Module 2 - A two file linkage
In this Module, you will build on what you have learnt from Module 1. You will learn how to undertake a linkage project in LinXmart involving two datasets – death records and hospital records.
The Module will help you gain confidence in operating the LinXmart software.
This module is part of a series of training modules that can be used with any LinXmart deployment. However, the user will require permissions to create Linkage Projects, load data for linkage extract linkage map results.
All required data files are available from the main self guideded training page.
Getting started
Step 1: Connect to the LinXmart web interface by typing in the LinXmart web address. You will see a screen similar to this:
Step 2: Ensure that your installation of LinXmart has a Data Provider called TRAINDP. To do this, click on the CATALOG tab at the top of the page.
If this Data Provider is not displayed, please follow the setup instructions for this series of training modules before continuing.
Creating a Linkage Project
Before data can be linked, we first define our project and include some metadata in the system – i.e. information about the data to be used and the linkages we wish to conduct.
Step 3: Click on the PROJECTS tab at the top of the web page.
This screen lists all the Linkage Projects that have been created in the system. We will create a new Linkage Project.
Step 4: Click on the Add Project link at the bottom of the page.
Enter the Linkage Project details in the fields provided. For the purposes of this Module, we will use XX_HOSP_DEATH.
As the value for Code must be unique in the system, change XX_ to something else (e.g. your initials) to create your Linkage Project. Click on SAVE when you have finished (see below).
Adding data to a Linkage Project
Step 5: Your XX_HOSP_DEATH project details page should be displayed.
You are presented with an empty project (as shown below).
Before data can be loaded, one or more Event Types need to be attached to the Project.
Step 6: Click on the Attach some event types link presented to you below the record stats for the empty project (shown below).
Click on the Event Types field.
You will be presented with a list of Event Types that are already known to (defined in) LinXmart.
Filter the list for our Data Provider by entering the text 'TRAINDP' in the search field. Then check the tickbox next to the Event Types TDP_DEATH amd TDP_HOSP.
Click OK to confirm the selection and the click Attach to attach these Event Types to the Linkage Project.
Linking these files
We have now added sufficient information into LinXmart to perform a two-file linkage of Hospital to Death records.
We will upload the data in two steps – first the Death data, then the Hospital data.
(Consider why we might upload Death records first?)
First, we upload the TDP_DEATH_clear.csv dataset found in the Training Data folder called TRAINDP Clear text datasets. You may wish to view this file in a text editor to see its contents. It is a comma delimited file contains personal identifiers and a unique record identifier. The format of this file (called the Import Format) has already been defined within the system.
Step 7: Click the Load icon on the TDP_DEATH row on the Event Type Data Sources page.
(To reach this page if you are not already on it, select Data Sources under the list of event types on the XX_HOSP_DEATH Linkage Project main page.)
Step 8: Click on the Choose File icon and select the TDP_DEATH_clear.csv file from the relevant Data folder.
Click Load. This may take a minute while the dataset is uploaded to the LinXmart server from your local machine.
You will be taken to a Load Data Confirmation page. Please ignore the Group by JLK option and select Load. This will begin the linkage process; its process can be monitored in real-time through the ACTIVITY tab.
Monitoring your load-linkage job
Step 9: To see how this load-come-linkage job is progressing, navigate to the Job Monitoring screen by clicking on the ACTIVITY tab at the top of the screen. Next select the ENVELOPES sub-tab. Information regarding the current job's status will be appear under ‘All Envelopes’.
The data in this file was internally linked or de-duplicated, identifying records within the file belonging to the same person. As there were no other datasets in the Project, this is the only linkage that took place. A green tick is the Status column indicates that the job ran to completion (green is good).
Uploading (and linking) the next file (Hospital records)
As this project is a linkage of Hospital to Death records, our next step is to include Hospital records in the mix!
Next, we upload the TDP_HOSP_clear.csv dataset found in the Training Data folder called TRAINDP Clear text datasets. There are about 25,000 records in this file. You may wish to view this file in a text editor to see its contents. It is a comma delimited file containing personal identifiers and a unique record identifier – all records are synthetic. The format of this file (called the Import Format) has already been defined within the system.
Step 10: To upload the data, first navigate to the Project dashboard (PROJECTS -> select XX_HOSP_DEATH) and then click on Data Sources.
Next, click the Load icon on the TDP_HOSP row on the Event Type Data Sources page (shown below).
Once again, select the Choose File icon, and select the TDP_HOSP_clear.csv file from the Training Data folder (shown below).
Click Load. This may take a minute while the dataset is uploaded to the LinXmart server from your local machine.
You will be taken to a Load Data Confirmation page. As previously, ignore the Group by JLK checkbox (if it is presented) and select Load.
This will begin the linkage process; its process can be monitored in real-time through the ACTIVITY tab.
Monitoring your Hospital load-linkage job
Step 11: To see how this next load-come-linkage job is progressing, navigate to the Job Monitoring screen by clicking on the ACTIVITY tab at the top of the screen. Select the ENVELOPES sub-tab.
Information regarding the current job's status will be appear under ‘All Envelopes’ (shown below).
A green tick is the Status column indicates that the job successfully ran to completion (green is good).
The file containing hospital data was both internally linked (i.e. to other hospital records) AND it was linked to other records in your Project. As your Project contained death records, this means that the hospital records were also linked to deaths.
Step 12: Click on the PROJECTS tab and select XX_HOSP_DEATH Project to go back to your Linkage Project dashboard.
The dashboard for the XX_HOSP_DEATH Project will now reflect the status of the project after the addition of the new Hospital data, showing you the records that were added and groups formed (unique individuals) (shown below).
You will notice in the Data Load History section, that there are two envelopes that belong to this Project – the first added Death records (which you did earlier), while the second added (and linked) Hospital records.
Step 13: In the Data Load History section, click on the report icon in the Options column for the most recent job that you just ran (shown below)
The Linkage Report details information on the changes made to the system for this particular Envelope, including storing a snapshot of the configuration used. Please take your time to read this report.
You can also click on the Notifications icon to the left of the Report icon to view other notifications that were sent out as part of the job.
Take your time to consider the information in these Reports.
Congratulations, you have just completed your second linkage!
Extension Exercise
As an additional step, navigate to the Quality Review screen to look at any quality issues that the system may have found with your linkage.
To do this, navigate to your project on the Project dashboard, and click on the Quality Review icon on the right-hand side (shown below).
Did any records break the Authoritative rule of Death records?
Having reviewed the Linkage Report for this Project, what might you do next?
Module 3 will assist you with the additional steps required for the Quality Review stage.