operations / skill
Duplicate File and Photo Report
Find byte-identical local files and photos, with reclaimable space estimates and no automatic cleanup.
At a glance
- Purpose
- Find byte-identical local files and photos, with reclaimable space estimates and no automatic cleanup.
- Runs in
- Local agent with shell
- Operating systems
- macOS, Linux
- Needs
- Runtime: Python 3.10+, standard library only; Needs internet: No; Needs accounts or API keys beyond your agent: No
- Files and data
- Reads files: Yes; Writes files: Yes; Sends data to other services: No
- Tool type
- Skill
- Delivery
- Download from Skills Outpost
- Version
- 0.1.0
- ZIP SHA-256
- 59267e8be3b8d087beb1c855752a847089b8be8ee9f7867287a21ee2b7bd0381
- Access
- Download from Skills Outpost
- Compatibility
- Python 3.10 or newer on macOS or Linux with POSIX directory descriptors and O_NOFOLLOW; Windows is unsupported and exits 2 before filesystem I/O. Standard library only. macOS SF_DATALESS is honored. An Agent Skills compatible harness that can run CLI tools is optional. No network, accounts or credentials.
- External costs
- None. The package is free under the MIT license and uses only local files and the Python standard library. It makes no network requests.
- Permissions
- Read directory metadata beneath caller-selected roots, including filtered entries; open eligible regular files for candidate hashes. The approval binds the fully resolved report destination, its directory and parent device/inode identities, every ancestor identity, and exact JSON bytes. Live writes walk directories with no-follow descriptor opens and create the report relative to the pinned directory. No directories are created; the report parent must already exist. All path components must be real directories, not symlinks. No environment configuration, credentials or network permissions.
- Usage terms
- MIT license. Copyright 2026 Zero Width Lab. macOS and Photos are trademarks of Apple Inc. This product is independent and is not affiliated with or endorsed by Apple. By Zero Width Lab (zerowidthlab.com)
Download from Skills Outpost
Get this version
FreeStatic source, format, secret-pattern and declaration checks with independent human source review.
- SHA-256 of the released ZIP
59267e8be3b8d087beb1c855752a847089b8be8ee9f7867287a21ee2b7bd0381
The creator shows this version's package files publicly: anyone can read SKILL.md and the other files under Files, without an account. The ZIP download needs the version in your Library (how it works).
Sign in to add this version to your library and follow its creator for free.
Sign in to get freeA creator follow is free, not a paid membership. Your email is not shared with the creator.
Files
Version 0.1.0, shown publicly by its creator, exactly as in this version's ZIP. ZIP SHA-256: 59267e8be3b8d087beb1c855752a847089b8be8ee9f7867287a21ee2b7bd0381. License: MIT (read it).
Each file has its own address. Read one as a web page at https://, or as JSON at https://; replace SKILL.md with any path below.
- manifest.json · 2.3 KiB
- LICENSE.txt · 1.0 KiB
- README.md · 5.4 KiB
- SETUP.md · 4.9 KiB
- SKILL.md · 6.0 KiB
- agents/duplicate-file-and-photo-report-agent.md · 4.2 KiB
- references/report-format.md · 4.1 KiB
- scripts/duplicate_file_report.py · 16.4 KiB
- source-package.json · 3.6 KiB
- tests/test_duplicate_file_report.py · 16.1 KiB
- tests/test_hygiene.py · 3.9 KiB
MIT
MIT LicenseCopyright (c) 2026 Zero Width Lab
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Version history
- Version 0.1.0 Current
No release notes for this version.
What it does
Duplicate File and Photo Report is a local command-line Agent Skill that finds byte-identical files and photos under paths you choose. An agent harness is software that can install the skill and run its Python command on your behalf. It needs Python 3.10 or newer on macOS or Linux and read access to the selected local folders. It is free under the MIT license. This listing delivers the complete package with its scripts.The scanner groups regular files by size, a sample SHA-256 hash of the first and last 64 KiB, and then a full SHA-256 hash. It reports each duplicate group's paths, byte size, modification times, and reclaimable bytes calculated as all copies minus one. You can use --min-size, --include-hidden, and --include-packages to control the scan. JSON output is available for further local analysis.
A report file is approval-gated: --dry-run previews the exact destination and report content digest, and a live --report write needs the matching --approve digest. The report is created only when the destination does not exist. The scanner never deletes, moves, or changes scanned files. The companion agent summarizes the largest opportunities and asks you to review any cleanup manually.
Symlinks, special files, and macOS package directories are skipped by default. Dataless iCloud files are never read because opening them could trigger a download; they are listed as not_downloaded. Unreadable or not-downloaded files make the result incomplete and exit 2 unless --allow-partial is set. Reading inside a Photos library is not a supported way to deduplicate that library.
Who it is for
People who want an offline inventory of duplicate local files or photos before deciding whether to free space, including users of agent harnesses that can run Python tools.Example tasks
Scan a Downloads folder and show the biggest duplicate groups.Compare two user-selected folders for byte-identical photos.
Scan files of at least 1 MiB and produce JSON output.
Preview a new report file, review its digest, and create it at the approved path.
List not-downloaded iCloud files without opening or downloading them.
Getting started
SKILL.md and the scripts can be read without an account: SKILL.md at https://
Run from the package directory, for example:
python3 scripts/duplicate_file_report.py scan ./Photos ./Downloads --min-size 1048576
python3 scripts/duplicate_file_report.py scan ./Photos --json
python3 scripts/duplicate_file_report.py scan ./Photos --report ./duplicate-report.json --dry-run
Review the dry-run result and pass its printed digest with --approve to create the report. Choose a new report path; existing files and symlinks are refused. Both --include-hidden and --include-packages are needed to enter a hidden package directory. Use --allow-partial only after reviewing the skipped-path list.
python3 scripts/duplicate_file_report.py scan ./Photos --report ./duplicate-report.json --approve DIGEST
Replace DIGEST with the dry-run digest. Use an existing real parent directory. Dry-run refuses an existing report destination. Use the physical path for symlinked ancestors (for example /private/tmp on macOS).
Limitations
Only byte-identical regular files are grouped; visually similar photos, resized images, edited images, and different encodings are not matched. The reclaimable estimate assumes one copy per group can be retained; actual disk savings can differ because of hard links, clones, compression, or filesystem allocation. Hidden entries and macOS package directories are skipped unless included explicitly. Scanning inside a Photos library is not a supported way to deduplicate that library. Dataless iCloud files and unreadable files prevent a complete result. Files can change between inspection and hashing, so review findings before acting.Version 0.1.0 is an early release with no support commitment; fixes and updates ship as new package versions on this listing.
SHA-256 compares observed bytes, not a stable filesystem snapshot. Symlinked ancestors are refused; symlink entries, including a ROOT that is itself a symlink, are skipped. Report parent directories must exist; none are created. Symlinked or non-directory ancestors of a scan ROOT are input errors (exit 2 even with --allow-partial); use the physical path for symlinked ancestors.
Data practices
Roots may be absolute or relative caller-selected paths; parent traversal (..) and NUL are rejected before scanning. Percent signs and backslashes are literal local filename characters; paths are never URL-decoded. Directory metadata is read to enumerate selected roots, including entries excluded by filters; hidden/package filters prevent descending excluded directories and opening excluded files. Candidate file bytes are read only for size groups with at least two files. Hashing streams chunks of at most 1 MiB, reads at most the observed size plus one growth-detection byte per full hash, and rejects changed files as unreadable/incomplete. No stdin is read.The approval binds the fully resolved report destination, its directory and parent device/inode identities, every ancestor identity, and exact JSON bytes. Live writes walk directories with no-follow descriptor opens and create the report relative to the pinned directory. No directories are created; the report parent must already exist. All path components must be real directories, not symlinks. No hosts are contacted and no names or content are sent to third parties. Reports and stdout reveal paths; review before sharing.