SETUP.md — Data Room Toolkit

# Setup 0.1.5

## Requirements

- Python 3.10 or newer
- A Microsoft Entra ID (formerly Azure AD) tenant where an administrator can grant application consent
- A SharePoint site and document library the application is allowed to access
- Outbound HTTPS to the destinations described below

No third-party Python packages are required.

## Application registration

1. Create or choose a Microsoft Entra ID application registration.
2. Record its tenant ID and client ID.
3. Create a client secret and store its value in a secure secret manager.
4. Add the Microsoft Graph application permissions described in
   [references/graph-permissions.md](references/graph-permissions.md).
5. Have an administrator grant consent.
6. When using `Sites.Selected`, grant the application access to only the
   intended site with the needed read or write role.

Portal navigation and the current `Sites.Selected` grant workflow must be
verified against Microsoft documentation before use.

## Permission trade-off

`Sites.Read.All` is simple for the read tools but can expose every site the
tenant permission covers. `Sites.ReadWrite.All` adds broad write capability for
the mirror. For a data room, prefer `Sites.Selected` with an explicit site grant
and the narrowest read or write role that supports the requested operation.
Confirm the exact scope of data each permission exposes against current
Microsoft documentation before relying on this trade-off description.

## Site, drive, and folder scope

Use `drive_id` when the document library drive is known. Otherwise supply
`site_id`; the tool resolves the site's default drive. Set `root_path` to scope
all commands below a library folder. Mirror also requires `--dest-path`, which
is joined below `root_path` and must already exist. The mirror refuses a missing
destination before any write; it never inspects or creates folders above it.
History paths are relative to `root_path`, with traversal rejected before requests. See
[references/config-schema.md](references/config-schema.md).

### How to find your `drive_id` or `site_id`

Both values come from Microsoft Graph, not from this package. One practical
path using [Graph Explorer](https://developer.microsoft.com/graph/graph-explorer),
signed in with an account that has at least read access to the target site:

1. Look up the site: `GET /v1.0/sites/{hostname}:/{server-relative-path}`,
   where `{hostname}` is your tenant's SharePoint host name (the first part
   of any site URL in your tenant) and the path is, for example,
   `/sites/YourSiteName`. The `id` field in the response is the `site_id`.
2. List its document libraries: `GET /v1.0/sites/{site-id}/drives`. Each
   entry's `id` field is a `drive_id`; the default library is also reachable
   directly at `GET /v1.0/sites/{site-id}/drive`.

Exact request paths and response shapes must be verified against current
Microsoft Graph documentation before use. A tenant administrator with access
to the SharePoint admin center may also be able to read these identifiers
from a site's properties there; the Graph calls above do not require admin
center access, only site read permission.

## Approved mirror workflow

Run the mirror in dry-run mode first. Optionally save its complete plan:

```bash
python3 scripts/mirror_dataroom.py --config dataroom.config.json --source ./export --dest-path Incoming --dry-run --plan-file plan.json
```

Show the complete plan and its digest and obtain explicit human approval. Then
use the approved digest directly, or use the approved plan file:

```bash
python3 scripts/mirror_dataroom.py --config dataroom.config.json --source ./export --dest-path Incoming --live --yes --plan-digest HEX
python3 scripts/mirror_dataroom.py --config dataroom.config.json --source ./export --dest-path Incoming --live --yes --plan-file plan.json
```

The dry run sends only read requests to Microsoft Graph and writes nothing to
SharePoint. It resolves the target drive and, when Graph reports it, the
owning site; checks the existing destination and folders beneath it and lists
only missing folders beneath it that a live run would create; checks each destination path; and marks each
file `create` (nothing at that path) or `replace` (an existing file, recorded
with its item ID and eTag). It prints the complete per-file action list and
names every existing destination file that a live run would replace. A live
mirror replaces those files. Locally, the dry run writes only the
`--plan-file` and `--audit-log` paths you name, creating a new file only at each path.

Live mode recomputes the whole plan before any write. If the drive or site,
destination root, options (verification mode, upload threshold, chunk size),
source paths, sizes, content hashes, destination paths, folders to create, or
any create/replace/refusal action differ from the approved plan, it prints the
differences and exits 1 before creating folders or uploading files. During the
writes, a create never replaces an existing item (documented). Every replace uses an
upload session with `If-Match: <approved eTag>` and is refused if the item
changed before session creation (documented) or before the final byte range
(checked by the tool). A change in the short window between that last check
and Microsoft's commit is not covered by any documented guarantee. See
[upload guarantees](references/graph-permissions.md#upload-guarantees).
Each local file is hashed again immediately before its first request; a file
whose size or SHA-256 changed is not sent. If local bytes or the replacement
eTag change before the final range, the session is cancelled with a
best-effort DELETE and the file is reported as failed. Empty replacements
are refused without writing because no final byte range can be sent.

## Environment variables

| Variable | Required | Purpose |
| --- | --- | --- |
| `GRAPH_TENANT_ID` | Yes | Tenant identifier. Treat as sensitive and never log it. |
| `GRAPH_CLIENT_ID` | Yes | Application identifier. Treat as sensitive and never log it. |
| `GRAPH_CLIENT_SECRET` | Yes | Client credential. Secret-bearing; never print or commit it. |
| `SHAREPOINT_DATAROOM_CONFIG` | No | Path to the default configuration JSON. |
| `SHAREPOINT_DATAROOM_OPERATOR` | No | Free-text audit note; defaults to `unspecified`. |
| `HTTPS_PROXY` / `https_proxy` | No | Honored by Python's standard library: HTTPS requests tunnel through this proxy. |
| `NO_PROXY` / `no_proxy` | No | Hosts that bypass that proxy. |
| `GRAPH_FAKE_RESPONSES` | Tests only | Fixture path. Never set outside test runs. |
| `GRAPH_FAKE_TRANSPORT_TEST_ACK` | Tests only | Must also be set, to the exact value the test suite uses, before `GRAPH_FAKE_RESPONSES` has any effect. This second signal exists so a leftover `GRAPH_FAKE_RESPONSES` in a shell profile or CI environment cannot silently divert Graph calls; never set outside test runs. |

Credentials come from environment variables only. The tools never load `.env`
files. Keep configuration free of credentials.

## External costs

None from this package. Normal Microsoft Graph service limits and throttling
still apply; bounded retry logic absorbs eligible transient responses.
Writes (folder creation, direct PUT, upload session, chunk, session cancel)
are retried only on HTTP 429 or on 503 with `Retry-After`. Another 5xx, a
transport error, or a 409/412 after an earlier attempt of the same write is
reported as "outcome unknown": the write may have completed, so run the dry
run again to see the current destination state.

## Network access

The tools make outbound HTTPS requests only to:

- `graph.microsoft.com`
- `login.microsoftonline.com`
- A tenant SharePoint content host designated by Microsoft Graph for a
  chunked-upload session or SHA-256 content-download redirect:
  `<tenant>.sharepoint.com` or `<tenant>-my.sharepoint.com`

The exact tenant content host is chosen by Microsoft Graph and is not
configured by this package. A Graph-issued upload URL or download redirect,
and every later redirect hop, is checked before it is contacted. It must use
HTTPS on the default port with no user information, and its host must be
`graph.microsoft.com`, `<tenant>.sharepoint.com` or
`<tenant>-my.sharepoint.com`, where `<tenant>` is exactly one DNS label. Bare
`sharepoint.com`, deeper subdomains, look-alike hosts and every other host are
refused. Sovereign-cloud SharePoint hosts are not supported. Ordinary
requests may reach only `graph.microsoft.com` and `login.microsoftonline.com`.

Proxies: the tools use Python's default `urllib` opener, which honors
`HTTPS_PROXY` or `https_proxy` (with `NO_PROXY` or `no_proxy` exemptions) and,
on macOS and Windows, the system proxy settings when those variables are
unset. If you set one, requests are tunnelled through that proxy you chose;
TLS still terminates at the hosts above and certificates are verified. A proxy
whose certificate authority your system trusts can inspect the traffic,
including credentials and uploaded content, so use only a proxy you trust.
`HTTP_PROXY` is not used because every request is HTTPS.

There is no telemetry.

## Removal

Delete the skill folder. This does not revoke the Microsoft Entra ID application
registration, remove its site grants, or invalidate its client secret. Revoke
those separately in the tenant if they are no longer needed.

## Limitations

- Application-permission flow only; delegated user authentication is not supported.
- Mirror is one way from local storage to SharePoint. There is no download or sync-back.
- Content search depends on SharePoint indexing latency.
- `Retry-After` parsing accepts only integer delta-seconds.
- SHA-256 verification re-downloads every uploaded file and uses bandwidth.
- Site-ID-to-drive resolution must be verified against Microsoft documentation.
- Current Graph direct-upload and chunk-size limits must be verified against
  Microsoft documentation before unusually large transfers.

Local --out, --plan-file (dry run) and --audit-log outputs create new files
only (0600) in existing parents; existing files and symlinked components are
refused. No local parent folders are created; audit logs do not append.
On platforms without directory-descriptor/no-follow support these writes fail closed.

One refused plan action blocks the whole live run: no file is uploaded.
An audit path is reserved before any remote write; failure records retain completed
entries and the interrupted file. A refused audit path prevents all uploads.
For symlinked output parents, use the physical path (for example /private/tmp).

Upload sessions default to 5 MiB (5,242,880 bytes = 16 x 327,680). Configuration and `--chunk-size` must be positive integer multiples of 320 KiB and strictly less than 60 MiB per request; invalid sizes are refused before authentication or any request. The final range may be shorter. See [Microsoft upload-session requirements](https://learn.microsoft.com/en-us/graph/api/driveitem-createuploadsession?view=graph-rest-1.0).

By Zero Width Lab (zerowidthlab.com)

License: MIT (read it).

Back to the listing