Work Wharton Research Data Services
One search box over four financial text corpora
News, social media, SEC filings and earnings transcripts, across different search backends, behind a single interface.
- Client
- Wharton Research Data Services
- Role
- Developer
- Period
- 2019 to 2023
- Focus
- Development, Search, Data Engineering
- Future Edge 50 2021 award winner
- 4 corpora unified
- 2 search engines behind one UI
Four very different text corpora, sitting behind two different search engines, needed to feel like one product. I built the application that does that.
Researchers search across news, social media, SEC filings and company transcripts without needing to know whether a given dataset lives in Elasticsearch, Solr or Postgres.
I also modeled the schema for the newest news dataset and did the data engineering to land it in both Elasticsearch and Postgres. That work is written up separately in Data Engineering at WRDS.
Built to outlive its first job
This is one half of the WRDS Text Analysis System, a 2021 Future Edge 50 award winner, paired with Text Analytics. It was deliberately designed so search could stand on its own later as a platform independent of the analytics half, rather than staying welded to it.
Screens
