Data flow
At a high level, a single run of Envelope Builder involves the following inputs and outputs. No data is transmitted from the local machine as part of this process — all files are read from, and written to, local storage (or a locally-mapped network) as determined by the operator.
Inputs
| Input | Description |
|---|---|
| Envelope name | An operator-supplied label for the run. |
| Project definition file | Created within LinXmart, this defines the fields expected, any cleaning/standardisation rules, and which fields (if any) require a hash or Bloom filter transform for a given linkage project. The project definition files are created by the linkage team and sent to the data custodian for use with Envelope Builder. |
| Source data file | The raw dataset supplied by the data custodian, typically a delimited text file (e.g. comma-separated) with a header row, mapped field-by-field to the project definition. |
| Key file | Only if the project definition requires hash or Bloom filter transforms — see Cryptographic Methodology. |
Processing
The operator maps columns in the source file to the fields specified in the project definition file, and may apply additional cleaning/standardisation transforms. Fields designated for privacy-preserving linkage are cryptographically encoded as described in Cryptographic Methodology; fields not designated for encoding are carried through as configured (which may include clear text), since Envelope Builder supports both clear-text and privacy-preserving linkage projects.
Output
The result is a single Envelope (zip) file containing the transformed dataset and some associated metadata, ready to be loaded into the LinXmart record linkage system. Depending on the project definition, this file may contain a mixture of clear-text fields and cryptographically encoded (hashed / Bloom filter) fields.
As the files in the Envelope zip are all text files, they can be reviewed by the Data Provider prior to the Envelope being released for linkage. The Envelope is typically shared with the linkage team using an established secure file transfer mechanism.