OpenDeID  Dataset

The OpenDeID corpus stands as a significant milestone as the first Australian-based gold-standard corpus specifically designed for patient de-identification purposes. This corpus holds immense value for the development and refinement of automated patient de-identification systems, whether they rely on rule-based algorithms or machine learning approaches. Comprising a total of 2,100 pathology reports, each report averaging approximately 717 tokens, the dataset draws from a pool of 1,833 cancer patients. Within this corpus, a meticulous annotation effort has resulted in the identification of 38,414 Protected Health Information (PHI) entities. Impressively, the inter-annotator agreement and deviation scores for all three de-identification settings demonstrate a high level of accuracy, measuring at 0.9464 and 0.9503, respectively. Worth noting is the fact that the corpus has been manually annotated with surrogate information, ensuring the absence of any identifiable patient data. This resource, meticulously crafted and rich in de-identified patient information, serves as a critical asset in advancing the development and evaluation of de-identification technologies and practices while upholding stringent privacy standards.

About HSA Biobank

The HSA biobank is a collaborative initiative based at the Lowy Cancer Research Centre at the University of New South Wales, Sydney, Australia. It aims at supporting researchers and clinicians for the advancement in the field of translational cancer research across Australia and internationally. The HSA Biobank houses all types of tumor tissues obtained from patients who have undergone surgery at one of the HSA hospitals and have provided their consent to the HSA Biobank. Please refer to this for more information.

http://www.tcrn.unsw.edu.au/hsa

Ethics approval

• This project is approved by the UNSW ethics panel and specific approval information can be found below.

• Principal Investigator: Jitendra Jonnagaddala (profile)

• Project Title: Automatic de-identification of unstructured pathology reports using deep neural networks

• Approval Number: UNSW HC17749

Dataset access fees

• Data access fee - A$12,500.00   (Excluding taxes)

• Payment via bank transfer to UNSW Sydney

Dataset Access Instructions

•  Fill out this form to obtain the corpus 

• Once the request is approved, please sign and return the SREDH Consortium membership, data usage, and project description forms that will be sent upon filling out the data request form above. 

• Pay data access and associated fees, if applicable 

• Download the dataset from the SREDH secure server. 

•  Submit a progress report every 6 months until the completion of the project

Access Criteria

• Available to researchers( academic and non-academic) for non-commercial purposes

• Researchers need to have experience in handling sensitive patient and training in ethics.

• Researchers are required to report bi-annually to the SREDH Consortium on any research outputs that arise.

• Any output that arises from this dataset needs to be reviewed by the data custodian (SREDH Consortium) before submission. 

Frequently Asked Questions

• Please refer to OpenDeID FAQs page.

 Selected publications

• • Jonnagaddala, J., Chen, A., Batongbacal, S., & Nekkantti, C. (2021). The OpenDeID corpus for patient de-identification. Scientific reports, 11(1), 19973. https://doi.org/10.1038/s41598-021-99554-9 

• • Chen, A., Jonnagaddala, J., Nekkantti, C., & Liaw, S. T. (2019). Generation of Surrogates for De-Identification of Electronic Health Records. Studies in health technology and informatics, 264, 70–73. https://doi.org/10.3233/SHTI190185 

• • Alla, N. L. V., Chen, A., Batongbacal, S., Nekkantti, C., Dai, H., & Jonnagaddala, J. (2021). Cohort selection for construction of a clinical natural language processing corpus. Computer Methods and Programs in Biomedicine Update, 1, 100024. https://doi.org/10.1016/j.cmpbup.2021.100024

• • Liu, J., Gupta, S., Chen, A., Wang, C. K., Mishra, P., Dai, H. J., Wong, Z. S., & Jonnagaddala, J. (2023). OpenDeID Pipeline for Unstructured Electronic Health Record Text Notes Based on Rules and Transformers: Deidentification Algorithm Development and Validation Study. Journal of medical Internet research, 25, e48145. https://doi.org/10.2196/48145