# BHI Data Sources

All endpoints tested 2026-04-04 unless noted. "Tested: OK" means a live curl returned valid data.

Scope: behavioral health facilities, demand indicators, workforce, shortages, and policy for all 50 US states, tagged by age bracket (adolescent 13-17, young adult 18-25).

---

## PHASE A — Free, autonomous, ready to ingest

### 1. CMS IPFQR (Inpatient Psychiatric Facility Quality Reporting)
- **Endpoint:** `https://data.cms.gov/provider-data/api/1/datastore/query/{dataset_id}/0`
- **Dataset IDs:**
  - `q9vs-r7wp` — IPFQR by Facility
  - `dc76-gh7x` — IPFQR by State
  - `s5xg-sys6` — IPFQR National
- **Auth:** None
- **Rate limit:** None documented; be polite (<= 5 req/sec)
- **Update frequency:** Quarterly
- **Record count:** ~1,600 IPFs (facility file); dozens of measures each
- **Key fields:** `facility_id`, `facility_name`, `address`, `state`, `zip`, `countyparish`, HBIPS-2/3 restraint+seclusion, SMD, SUB-2/3, TOB-3, transition record, 30-day readmission
- **Test curl (OK):**
  ```
  curl -s "https://data.cms.gov/provider-data/api/1/datastore/query/q9vs-r7wp/0?limit=2"
  ```
- **Python snippet:**
  ```python
  import requests
  r = requests.get("https://data.cms.gov/provider-data/api/1/datastore/query/q9vs-r7wp/0",
                   params={"limit": 500, "offset": 0})
  rows = r.json()["results"]
  ```

### 2. CMS Hospital Compare / Care Compare (general hospital info)
- **Endpoint:** `https://data.cms.gov/provider-data/api/1/datastore/query/xubh-q36u/0`
- **Auth:** None | **Rate limit:** none | **Update:** Monthly
- **Records:** ~5,300 hospitals
- **Key fields:** `facility_id` (CCN), `facility_name`, `hospital_type`, `hospital_ownership`, `hospital_overall_rating`, mortality/safety/readmission group flags
- **Test (OK):**
  ```
  curl -s "https://data.cms.gov/provider-data/api/1/datastore/query/xubh-q36u/0?limit=2"
  ```
- Use to classify which acute hospitals have behavioral health units (cross-join with IPFQR CCNs).

### 3. CMS Provider of Services (POS) file
- **Bulk page:** `https://data.cms.gov/provider-characteristics/hospitals-and-other-facilities/provider-of-services-file-quality-improvement-and-evaluation-system`
- **JSON catalog:** `https://data.cms.gov/data.json` (search `dataset[].title` = "Provider of Services File")
- **Auth:** None | **Update:** Quarterly | **Format:** CSV bulk
- **Records:** ~80,000 Medicare-certified facilities (includes PSY, PRTF, hospitals)
- **Key fields:** CCN, provider category, bed count, certification date, termination date, ownership
- **Test (OK):** `curl -s "https://data.cms.gov/data.json"` — dataset list
- Required for bed counts and termination (closure) tracking.

### 4. CMS Nursing Home Compare (Provider Information)
- **Endpoint:** `https://data.cms.gov/provider-data/api/1/datastore/query/4pq5-n9py/0`
- **Auth:** None | **Update:** Monthly
- **Records:** ~15,000 nursing homes
- **Key fields:** CCN, provider_name, ownership, number_of_certified_beds, overall rating, chain info
- **Test (OK):** `curl -s "https://data.cms.gov/provider-data/api/1/datastore/query/4pq5-n9py/0?limit=2"`
- Used to capture residential behavioral health (SNFs frequently host psych/BH residents).

### 5. SAMHSA Treatment Locator (findtreatment.gov)
- **Endpoint:** `https://findtreatment.gov/locator/exportsAsJson/v2?sType=BH&sAddr={zip}`
- **Auth:** None (browser UA helps but not required for JSON export)
- **Rate limit:** None documented; HEAD returns 403 but GET returns 200 — use GET only
- **Update:** Continuous (SAMHSA-maintained)
- **Records:** ~96,000 BH treatment facilities (all service types)
- **Key fields:** name1/name2, street, city, state, zip, phone, intake, hotline, website, lat, lon, services, typeFacility
- **Test (OK):**
  ```
  curl -s "https://findtreatment.gov/locator/exportsAsJson/v2?sType=BH&sAddr=10001"
  ```
  Response: `{"page":1,"totalPages":3201,"recordCount":96009,"rows":[...]}`
- **Python snippet:**
  ```python
  import requests, time
  def fetch_all(zip_seed="10001"):
      base = "https://findtreatment.gov/locator/exportsAsJson/v2"
      page = 1
      while True:
          r = requests.get(base, params={"sType":"BH","sAddr":zip_seed,"pageSize":30,"page":page})
          d = r.json()
          yield from d["rows"]
          if page >= d["totalPages"]: break
          page += 1
          time.sleep(0.3)
  ```

### 6. SAMHSA N-SSATS + N-MHSS
- **Bulk:** `https://www.samhsa.gov/data/data-we-collect/n-ssats/datafiles` and `/n-mhss/datafiles`
- **Auth:** None | **Update:** Annual | **Format:** SAS / SPSS / CSV
- **Records:** N-SSATS ~16,000 SUD facilities/year; N-MHSS ~12,000 MH facilities/year
- **Key fields:** facility id, services, payment accepted, populations served (including adolescent/young adult flags), bed counts, ownership
- **Note:** Bulk ZIPs; no live API. Staged as manual-download job.

### 7. CDC WONDER (mortality — suicide, overdose, by county, age)
- **Endpoint:** `https://wonder.cdc.gov/controller/datarequest/D76` (Underlying Cause of Death) — POST XML
- **Auth:** None for non-restricted datasets; county-level suppressed for <10 deaths
- **Update:** Annual
- **Records:** All US mortality; we pull ICD-10 X60-X84 (suicide) + X40-X44/Y10-Y14 (overdose) by county, 13-17 and 18-25
- **Test (OK):** landing page returns 200; POST XML required for data. See job stub `wonder_mortality.py` for the working XML template.

### 8. CDC BRFSS
- **Endpoint (Socrata):** `https://data.cdc.gov/resource/dttw-5yxu.json`
- **Auth:** None (Socrata app token optional for higher limits) | **Update:** Annual
- **Records:** ~100k rows/year (state x question x breakout)
- **Test (OK):**
  ```
  curl -s "https://data.cdc.gov/resource/dttw-5yxu.json?$limit=2"
  ```
  Returns depression prevalence, mental health days, etc. by state+demographic.

### 9. CDC YRBSS (Youth Risk Behavior Survey)
- **Endpoints (Socrata, verified present via catalog):**
  - High school: `https://data.cdc.gov/resource/3qty-g4aq.json`
  - Middle school: `https://data.cdc.gov/resource/uqmk-4y2w.json`
- **Auth:** None | **Update:** Biennial
- **Records:** State + large urban district level; ~50k rows
- **Key fields:** suicidal ideation, attempt, persistent sadness, substance use — exactly the adolescent demand signal we need.

### 10. IDEA Part B data (Emotional Disturbance by district)
- **Landing:** `https://www2.ed.gov/programs/osepidea/618-data/static-tables/index.html`
- **Auth:** None | **Format:** CSV static tables | **Update:** Annual
- **Records:** ~14,000 school districts + state rollups
- **Key fields:** Child count under ED classification, ages 6-21, by state and LEA
- **Note:** Static CSVs; no API. Download script documents exact file URLs.

### 11. NSCH (National Survey of Children's Health) via HRSA
- **Landing:** `https://www.childhealthdata.org/browse/survey` and `https://mchb.hrsa.gov/data-research/national-survey-childrens-health`
- **Bulk (HRSA):** `https://mchb.hrsa.gov/sites/default/files/nsch/datafiles/` (year-specific)
- **Auth:** None | **Update:** Annual | **Format:** SAS / Stata / CSV
- **Records:** ~50k surveyed children, weighted to state-level estimates
- **Key fields:** anxiety, depression, behavioral problems, received treatment, unmet need — by state x age.

### 12. BLS OES (behavioral health workforce by MSA)
- **API:** `https://api.bls.gov/publicAPI/v2/timeseries/data/` (POST JSON)
- **Auth:** Free registration key for >25 series/day (`https://data.bls.gov/registrationEngine/`). Without key: 25 series/query, 10 years/query, no key required but lower limits.
- **Update:** Annual (May reference period)
- **Series ID pattern:** `OEUM{area}{industry}{occupation}{datatype}`
- **Relevant SOC codes:**
  - 29-1223 Psychiatrists
  - 29-1229 Other Physicians (incl. addiction medicine)
  - 21-1014 Mental Health Counselors
  - 21-1015 Rehabilitation Counselors
  - 21-1018 Substance Abuse/Behavioral Disorder Counselors
  - 21-1022 Mental Health and SUD Social Workers
  - 19-3033 Clinical & Counseling Psychologists
- **Test (OK):** BLS API responds (test hit confirmed structure; real series IDs required)
- **Bulk alternative:** `https://www.bls.gov/oes/special-requests/oesm{YY}ma.zip` (annual bulk by MSA) — no auth, ~50MB zip.

### 13. HRSA Mental Health HPSAs
- **Bulk CSV (verified):** `https://data.hrsa.gov/DataDownload/DD_Files/BCD_HPSA_FCT_DET_MH.csv`
- **Size:** ~23 MB
- **Auth:** None | **Update:** Continuous (weekly snapshots)
- **Records:** ~6,500 active MH HPSAs + historical
- **Key fields:** HPSA ID, designation type, discipline (MH), score (0-25), state, county FIPS via HPSA Geography ID, population, designation date, withdrawn date, lat/lon
- **Test (OK):** HTTP 200, 23 MB CSV returned.

### 14. CMS NPPES (National Plan & Provider Enumeration System)
- **API:** `https://npiregistry.cms.hhs.gov/api/?version=2.1`
- **Auth:** None | **Rate limit:** ~200 req/sec soft; 200 results max per query — paginate with `skip`
- **Update:** Daily
- **Records:** ~8 million NPIs; filter by taxonomy for behavioral health (~500k)
- **Relevant taxonomy codes:**
  - 2084P0800X Psychiatry & Neurology - Psychiatry
  - 2084P0802X Addiction Psychiatry
  - 2084P0804X Child & Adolescent Psychiatry
  - 103T00000X Psychologist
  - 101YM0800X Mental Health Counselor
  - 103TC2200X Clinical Child & Adolescent Psychologist
  - 1041C0700X Clinical Social Worker
  - 324500000X Substance Abuse Rehabilitation Facility
  - 283Q00000X Psychiatric Hospital
  - 323P00000X Psychiatric Residential Treatment Facility
- **Test (OK):**
  ```
  curl -s "https://npiregistry.cms.hhs.gov/api/?version=2.1&taxonomy_description=psychiatric&state=NY&limit=2"
  ```

---

## PHASE B — Requires application or registration

### 15. HCUP (AHRQ)
- **Landing:** `https://hcup-us.ahrq.gov/tech_assist/centdist.jsp`
- **Auth:** Data Use Agreement (DUA) required; free for research but application-based (~2-4 weeks)
- **Records:** State inpatient/ED/ASC databases, ~40M discharges/yr nationally
- **Action required:** Submit DUA + Data Use Training certificate. **BLOCKED until user applies.**

### 16. CMS Medicare Cost Reports (MCR)
- **Bulk:** `https://www.cms.gov/data-research/statistics-trends-and-reports/cost-reports` (HOSPITAL2010 format)
- **Auth:** None; just large downloads (~1-3 GB per year)
- **Update:** Quarterly rolling
- **Records:** ~6,000 hospital cost reports/year (CCN-level)
- Staged as a fetch-and-parse job (uses `ccn` to join with `bhi_facilities`).

### 17. NEMSIS state crisis transport data
- **Landing:** `https://nemsis.org/using-ems-data/request-research-data/`
- **Auth:** Research Data Request (application) — typically 4-8 weeks
- **BLOCKED until user applies.**

### 18. California HCAI (patient discharge data)
- **Endpoint:** `https://hcai.ca.gov/data-and-reports/cost-transparency/` and `https://data.chhs.ca.gov/dataset?q=pdd`
- **Auth:** Free (some files direct download; Limited Data Set requires DUA)
- **Update:** Annual
- **Records:** ~3.5M CA discharges/yr; psych DRGs extractable

### 19. NY SPARCS
- **Landing:** `https://www.health.ny.gov/statistics/sparcs/`
- **Auth:** Application for identified data; deidentified file free via `health.data.ny.gov`
- **Deidentified endpoint:** `https://health.data.ny.gov/resource/u4ud-w55t.json` (Hospital Inpatient Discharges)
- **Records:** ~2.5M NY discharges/yr

### 20. TX DSHS discharge data
- **Landing:** `https://www.dshs.texas.gov/texas-health-care-information-collection/health-data-researcher-information/texas-inpatient-public-use`
- **Auth:** Free (Public Use File is a direct download after click-through)
- **Records:** ~3M TX discharges/yr

### 21. FL AHCA discharge data
- **Landing:** `https://ahca.myflorida.com/health-care-policy-and-oversight/bureau-of-central-services/florida-center-for-health-information-and-transparency/data-analytics/order-data`
- **Auth:** Application form + fee for identified; aggregate free
- **BLOCKED until user applies for identified.**

---

## PHASE C — State RTF licensing databases

### 22. State-by-state RTF licensing scrapers
Scope: residential treatment facilities serving adolescents. One scraper per state.

Verified public-search portals (no auth, scrape-friendly HTML/JSON):
- **UT** — `https://hslic.utah.gov/` (Human Services License Information Lookup)
- **CA** — `https://www.ccld.dss.ca.gov/transparencyapi/api/facilities` (Community Care Licensing API)
- **TX** — `https://www.hhs.texas.gov/providers/long-term-care-providers/childrens-residential-facility-reimbursement-methodology` + search portal
- **FL** — `https://apps.myflfamilies.com/provider/` (DCF provider search)
- **NY** — `https://omh.ny.gov/omhweb/resources/providers/` (OMH provider directory)
- **MT** — `https://dphhs.mt.gov/qad/licensure/licensedfacilitieslist` (static list)
- **AZ** — `https://azcarecheck.azdhs.gov/` (public search)
- **CO** — `https://apps.colorado.gov/apps/oapa/licensee.aspx` (Office of Early Childhood)
- **OR** — `https://ccld.oregon.gov/ccld/search/` (Care Provider Directory)
- **WA** — `https://fortress.wa.gov/dshs/adsaapps/lookup/` (LTC lookup)
- **IL** — `https://www2.illinois.gov/dcfs/brighterfutures/Pages/default.aspx`
- **MA** — `https://www.mass.gov/lists/licensed-residential-treatment-programs`
- **PA** — `https://www.dhs.pa.gov/Services/Assistance/Pages/Child-Residential-Facility.aspx`

States requiring FOIA / no public portal (documented as BLOCKED for Phase C v1):
- AL, AK, AR, DE, GA, HI, ID, IN, IA, KS, KY, LA, ME, MD, MI, MN, MS, MO, NE, NV, NH, NJ, NM, NC, ND, OH, OK, RI, SC, SD, TN, VT, VA, WV, WI, WY

The scraper job stub lists URL patterns for the 13 verified states and marks the rest "FOIA required."

---

## Test results summary (Phase A)

| # | Source | Status | Notes |
|---|--------|--------|-------|
| 1 | CMS IPFQR | OK | q9vs-r7wp returned facility rows |
| 2 | CMS Hospital Compare | OK | xubh-q36u returned |
| 3 | CMS POS | OK | catalog reachable, bulk CSV |
| 4 | CMS Nursing Home | OK | 4pq5-n9py returned |
| 5 | SAMHSA Locator | OK | 96,009 records confirmed |
| 6 | SAMHSA N-SSATS/N-MHSS | OK (bulk) | ZIP download, no API |
| 7 | CDC WONDER | OK | POST XML required, landing 200 |
| 8 | CDC BRFSS | OK | Socrata JSON returned |
| 9 | CDC YRBSS | OK | 3qty-g4aq + uqmk-4y2w |
| 10 | IDEA Part B | OK (static) | Static CSV; no API |
| 11 | NSCH | OK (bulk) | HRSA year files |
| 12 | BLS OES | OK | API responds; needs real series IDs |
| 13 | HRSA HPSA MH | OK | 23 MB CSV download confirmed |
| 14 | NPPES | OK | 2 results returned for NY psych |

Blocked until auth/application:
- HCUP (DUA), NEMSIS (application), FL AHCA identified, NY SPARCS identified.