A self-hosted platform that ingests articles from Bangladeshi news sources and runs them through a multi-stage AI pipeline (discover → fetch → parse → normalize → classify → extract → deduplicate) to produce structured incident records — location, casualties, entities, and response actions — surfaced through a public map/explorer and an internal admin dashboard.
Reports of disasters, accidents, and public-safety incidents in Bangladesh are scattered across dozens of news outlets in unstructured, duplicate-heavy articles. There's no single, structured, queryable source of truth for what happened, where, and how severe it was.
Built a Bun/TypeScript monorepo with independently scalable BullMQ workers that scrape and process articles end-to-end: fetching raw HTML (archived to MinIO), extracting structured facts via Gemini LLMs, deduplicating near-identical reports using vector similarity search (Qdrant) plus LLM-assisted merging, and storing the results in PostgreSQL/PostGIS for geospatial querying. A Next.js public portal visualizes trends and impact, while an admin app manages sources, review, and AI pipeline health.
Converts fragmented, duplicate news coverage into a single trustworthy, geolocated incident dataset — with full AI cost/telemetry tracking — enabling researchers, journalists, and responders to see incident trends, human impact, and geographic hotspots at a glance instead of manually cross-referencing news articles.
