Healthcare FHIR Data Ingestion Framework
The framework transforms complex FHIR resources into analytics-ready datasets with PySpark, layered data modeling, and patient-history features for healthcare analytics and prediction.
From Healthcare Resources to Analytical Insight
The FHIR ingestion and feature engineering framework uses PySpark to connect healthcare data with downstream analytics. The work spans resource processing, analytical data modeling, and patient-history features—the foundation needed before reporting or predictive modeling can begin.
Project focus: turning resource-level healthcare data into structured datasets and meaningful patient context for analytical use.
Explore the implementation on GitHub
The Engineering Challenge
Healthcare questions rarely stop at a single resource. Understanding a patient’s history requires connecting patient information, encounters, procedures, and conditions, then translating those records into features such as prior admissions and chronic condition flags.
The framework addresses this data engineering problem through three connected responsibilities: process FHIR resources, organize the resulting entities for analysis, and derive features that downstream consumers can use.
Architecture: Bronze, Silver, and Gold
The repository’s workflow moves from raw resources through structured healthcare entities into an encounter-level analytical dataset.
| Layer | Data Flow | Purpose |
|---|---|---|
| Bronze | Raw resources | Source data for transformation. |
| Silver | Patient, encounter, and condition datasets | Structured healthcare entities for analysis. |
| Silver condition processing | Condition standardization, risk mapping, and rollups | Condition context prepared for downstream use. |
| Gold | Encounters combined with patient and condition context | An analytical dataset connecting the prepared entities. |
The architectural value is in separating responsibilities. Resource transformations establish the entities, condition processing adds domain context, and the final dataset brings those outputs together. This gives the analytical workflow an explicit path back through the preparation stages.
Implementation Deep Dive
Processing FHIR Resources with PySpark
The framework includes ETL pipelines for patient, encounter, procedure, and condition resources. This required working across the healthcare resource model: patient information supplies context, encounters describe interactions with care, and conditions and procedures add clinical detail.
PySpark provided the processing framework for the transformations. The transformations prepare data for questions that span a patient’s history and make that context available beyond individual source records.
Engineering Patient-History Features
Feature engineering pipelines turn healthcare history into analytical signals.
| Feature Family | Derived Features | Downstream Value |
|---|---|---|
| Prior admissions | Features describing previous admissions | Historical utilization context. |
| Chronic conditions | Flags representing chronic condition history | Persistent health context. |
| Encounter history | Features describing encounter trends | Patterns across interactions with care. |
These features bridge ingestion and analytics. Their purpose is to express patient context in a form that can support both exploratory analysis and the companion readmission modeling workflow.
Engineering Capabilities Demonstrated
The project combines ingestion and transformation pipelines, patient-level feature engineering, and analytical data preparation. It demonstrates three connected engineering capabilities:
- Healthcare data engineering: processing FHIR resources while retaining their analytical context.
- Data modeling: organizing raw and transformed data into a layered workflow.
- Feature engineering: translating clinical and administrative history into reusable inputs for analysis.
The framework connects source data representation with the structures required by analytical consumers.
Deliverables and Analytical Value
The framework delivers structured healthcare datasets and patient-history features for outcomes analysis, operational questions, and predictive modeling. It connects resource processing with an analytical use case through a shared preparation workflow.
The companion readmission project builds on this foundation. Together, the two projects carry healthcare data from ingestion into applied analytics, with a distinct focus on the engineering responsibilities at each stage.
Technical Stack
| Area | Technologies and Methods |
|---|---|
| Processing | Python, PySpark, ETL pipelines |
| Healthcare Data | FHIR patient, encounter, procedure, and condition resources |
| Architecture | Bronze, Silver, and Gold layers |
| Analytical Modeling | Patient and encounter context, condition rollups |
| Feature Engineering | Prior admissions, chronic condition flags, encounter history trends |
Explore the Work
View the source repository for the implementation and workflow.
Continue with 30-day Patient Readmissions Predictive Model to see how the prepared data supports classification.