r/datasets • • 17h ago

dataset US WARN layoff notices by state (California, Florida, New York, Tennessee, Maryland) as free cumulative CSVs from the state labour departments

6 Upvotes

Money Tracker Daily publishes every WARN layoff notice filed with five state labour departments (California EDD, FloridaCommerce, New York State Department of Labor, Tennessee Department of Labor and Workforce Development, Maryland Department of Labor) as free cumulative CSVs, licensed CC BY 4.0.

Nothing is removed when a state drops a notice from its own list, so each file works as a time series. Each state also has a page per employer, county or city, industry or reason, and month.

https://moneytrackerdaily.com/trackers/layoffs/california

(I run the site. Happy to answer questions about how the data is collected.)


r/datasets • • 19h ago

dataset Weekly counts of software jobs by job type, skills and years asked, from about 255 company career boards (CSV, CC BY 4.0)

Thumbnail github.com
4 Upvotes

I've been counting public tech job ads every week and publishing the counts as CSVs. It's three weeks in now, so I'm sharing it in case it's useful to someone.

Each week gets a folder with three files:

- job_types.csv: jobs counted, jobs long enough to read, and employers, for 5 job types (AI, machine learning, DevOps, software engineering, entry level software)

- skills.csv: which skills the ads name, how many list each one as required, and the share

- experience.csv: years of experience required, for jobs with an early career title and for all software jobs

How it's counted: ads from about 255 employer boards (Greenhouse, Ashby, Lever, Workable) plus a few job sites. The same title at the same employer counts once. Skills are matched with fixed rules against a list of about 250, no language model.

Caveats: the boards lean towards US employers, AI companies and fast growing firms, so it's not the whole market. It counts ads, not hires. Three weeks isn't a trend yet. It updates every Monday.

Latest week: https://github.com/DerrickWawerumuturi/glassbox-job-counts/tree/main/2026-10-19

I also wrote up one thing it shows (629 of the 4,536 software jobs that week were entry level, and 274 of those were internships): https://seeglassbox.com/market/entry-level-software/is-entry-level-dead?utm_source=reddit&utm_campaign=datasets

Disclosure: I made this, it's my project (Glassbox). The data is CC BY 4.0 and the story needs no sign up. I'd like to hear what you'd want counted next.


r/datasets • • 8h ago

dataset Free sample of UK and Irish bookmaker odds from OddsRelay, with outcomes matched across bookmakers (CSV, JSON)

2 Upvotes

We've put a free sample of UK and Irish bookmaker odds on Hugging Face, with the outcomes already matched across bookmakers. It's one pre-match snapshot from OddsRelay, our odds API, taken on 11 Oct for events in the next 24 hours.

https://huggingface.co/datasets/oddsrelay/uk-bookmaker-prices-sample

If you're building a model or an odds screen on UK prices, you'll know every bookmaker names teams and outcomes its own way. The matched table puts all those prices under one name per outcome, 15,820 of them. There's a raw table too if you'd rather do the matching yourself.

Football, tennis, horse racing, darts and more, up to 15 events a sport. CSV and JSON under CC BY 4.0, with a data dictionary, and everything joins on event_id and bookmaker_key.

There's no history or closing prices, in the sample or in the feed. Pre-match only. The sample also leaves the exchanges out.

The feed itself has 60+ UK and Irish bookmakers, with Betfair, Smarkets, Matchbook and BETDAQ lays next to the back prices. If you want current prices on your own events, the Free plan is 50,000 tokens for 14 days with no card.


r/datasets • • 8h ago

dataset YouTube monetization signals by country (50 countries, 3,080 channels, CC BY 4.0)

2 Upvotes

I run a small tool that guesses whether a YouTube channel is in the Partner Program from what's public on the page. YouTube stopped showing that status publicly in late 2023, so the tool works from other signals. I put the per-country counts into a dataset.

One row per country: channels checked, how many came out likely monetized, possibly, likely not, and couldn't tell, plus median subscribers. Countries with fewer than 30 channels are left out.

The channels come from each region's trending videos (YouTube Data API, 111 regions, 10 categories), so they skew big. A country's number is about the big channels we checked, not the country as a whole.

About 20% we couldn't call either way, and that column is in there.

One thing I got wrong: the bottom eight countries (Kazakhstan, Tanzania and Bangladesh at 0 of 60) are exactly the eight where YouTube doesn't offer channel memberships, and the September rule needed the memberships Join button for a "likely" verdict. That rule has since been changed. This file is the September snapshot; the repo has a comparison of the two.

GitHub: https://github.com/supertj/youtube-monetization-by-country

Live version (updated weekly, new rule): https://monetizationkit.com/data/youtube-monetized-channels-by-country/

If something looks off in a country row, I'd like to hear it.


r/datasets • • 3h ago

request Looking for devs who need US stock and fund data

1 Upvotes

Quick background: I worked with most of the US fundamentals providers and kept hitting the same issues. Blackbox normalization, numbers that didn't match the actual filing, restatements overwriting history, and pricing that jumps the moment your project becomes commercial. So I started parsing SEC EDGAR myself, and it turned into this within 2 years of development.

Here's what's in it right now.

1. Coverage: everything that files with SEC EDGAR

- ~8,300 operating US stocks, OTC names included

- ~5,500 delisted companies with full fundamentals, so backtests don't suffer survivorship bias

- ~19,500 funds: ETFs, mutual funds and closed-end funds

- Lookup by ticker, CIK, CUSIP or FIGI, plus former names and prior tickers

2. Clean fundamentals, point-in-time correct if you care about that

Income statement, balance sheet and cash flow, normalized from XBRL. 10-K/10-Q filers plus 20-F and 40-F foreign filers.

Every period carries a `dateFiled` (the original filing date, not the latest amendment), `acceptedAt` and a `sources` block keyed by SEC accession number. Every filing that touched the period is listed, and when a later filing changed a number, the old value is also served.

Real example, United Airlines FY2017 net income:

- 10-K filed 2018-02-22: $2,131M

- 10-K filed 2019-02-28: restated to $2,144M

- 10-K filed 2020-02-25: restated to $2,143M

All three are in the response, so you can rebuild what was known on a given date without lookahead bias. Add includeSourceDetails=true and each entry also tells you which XBRL concept it came from and the unit it was filed in and when that filing was accepted. If you'd rather skip my normalization entirely, `/as-reported` returns the raw XBRL facts, custom extensions included.

3. Footnote extractions

The stuff that never makes it onto the face of the statements, pulled from the tagged footnotes:

Debt by individual tranche (coupon, maturity, fair value, conversion price), credit facilities, leases, stock comp, customer and supplier concentration, pensions, restructuring programs, buybacks, fair value hierarchy, REIT property schedules, backlog/RPO, supplier finance programs, subsidiary financials, workforce cost, plus revenue segmentation (geography and product) and business segments.

Example: Qorvo's Apple share of revenue went from 30% in FY2021 to 50% in FY2026. That's one call, and every point comes with the date it was filed.

4. Key metrics that know what sector they're looking at

/api/financials/key-metrics returns a general block for every company, then a sector block and an industry block picked from the company's classification. There are 13 sector views and 8 industry views. Banks get NIM, efficiency ratio and loan-to-deposit. Insurers get combined ratio and float. REITs get FFO and AFFO. Retailers get revenue per store. Software gets deferred revenue, billings and RPO. Pharma gets cash runway in quarters. Gross margin on a bank doesn't mean anything, so the sector block gives you what does.

5. Daily fund holdings

Holdings for 5,000+ ETFs, pulled once a day straight from each issuer's own website (CSV, Excel, JSON, HTML, even PDF, all normalized into one shape). That's a lot fresher than the quarterly N-PORT filings. When I checked ARKK today, its holdings were dated today. A `reportDate` parameter gives you the snapshot as of a past date (about 30 days of history per fund), and bond funds come with coupon and maturity.

6. Deeper fund data

For anyone doing serious fund work, built from the SEC fund filings (N-PORT, N-CEN, prospectuses): full quarterly holdings, reverse lookup (which funds hold a given stock), overlap between two funds, flows, portfolio changes, fee schedules, fee and cost analysis, service provider history, funds by adviser, structure, fund health, performance and allocation breakdowns.

Example: ARKK's N-CEN data shows about $2.0M in brokerage commissions for the year ending July 2026, split by broker (the largest got ~$508k), plus what share of creations and redemptions happened in kind.

7. Congressional trades

House and Senate STOCK Act disclosures, going back to the first reports (Senate 2012, House 2014). Right now that's 8,400+ House reports and 2,400+ Senate reports from 398 House members and 82 senators. Each filer is matched to their official Bioguide ID, amount ranges come as min/max numbers, ownership is split into self/spouse/child/joint, and tickers are resolved with a confidence flag so you can see when a match is shaky. Query by company, by member, or as a market-wide feed. Paper filings are listed and flagged as unparsed rather than silently missing.

Also in there: insider transactions, 13F institutional ownership, beneficial owners, executive comp, filing search and section extraction (pull just Item 1A or Item 7 out of a 10-K), earnings and dividend history, and prices.

Pricing: there's a free tier to try it out. Commercial use starts at $19/month, and the top plan at $99/month includes everything above. No pricing cliff the moment you go commercial; that was the whole point.

What I'm asking for

I'm a solo founder. If you use stock or fund data in anything, I'd love for you to try it and tell me:

- What's missing that would unblock what you're building

- Any number that looks wrong. Send me the ticker and period and I'll trace it back to the filing and fix it

Feedback like that shaped most of the list above, and right now it can still change the roadmap quickly.

Website: developer.stockfit.io

Docs: api.stockfit.io/docs


r/datasets • • 6h ago

dataset [Dataset] Structural audit of 137 official government PDF forms (19,512 form fields across IRS, USCIS, SSA, VA, State Dept, CA Courts): XFA layers, encryption bits, field counts, and naming readability (CC BY 4.0 CSV)

1 Upvotes

We collected the official blank PDFs for 137 government form entries (134 analyzed across 133 distinct PDF files, totaling 63.4 MB, 628 pages, and 19,512 terminal form fields) and ran a structural inspection using pypdf 6.6 to measure how government agencies actually construct and publish PDF forms.

The full dataset is available as a static, no-JS CSV under CC BY 4.0: - Direct CSV Download (137 rows × 26 columns): https://justfill.app/data/justfill-government-forms-fillability-2026-09-02.csv - Methodology, Caveats & Sortable Table: https://justfill.app/research/are-government-forms-fillable

Schema (26 Columns):

Column Type Description
slug string Canonical form identifier (e.g., w-9, i-129, fl-150, form-2848)
formName string Official form number/title
issuer string Full issuing agency name
group string Agency cohort (IRS, USCIS, SSA, VA, State Dept, DOL, CMS, California courts, etc.)
country string Jurisdiction (United States, United Kingdom, Germany, Spain, India)
status enum ok (134), blocked (2 — HTTP 403 outside India), no_pdf (1 — online form only)
pdfUrl string Direct agency URL fetched on collection date (2026-09-02)
pages int Page count of the official blank PDF (total: 628, median: 3.0)
sizeKb int File size in KB (largest: State Dept DS-64 at 3,150 KB)
fields int Total terminal AcroForm input fields (excluding pushbuttons)
textFields, checkboxes, radios, choices, signatures int Breakdown by widget type (14,207 text, 5,029 checkboxes, 202 dropdowns, 58 radio groups, 16 signatures)
xfa bool Whether /AcroForm contains an /XFA XML package
encrypted bool Whether the file carries owner encryption
modifyDenied, annotateDenied bool Parsed PDF permission bits
textLayer bool Whether page 1 yields >= 50 extractable characters
producer, pdfVersion, edition string PDF producer metadata, version (1.5–1.7), and regex-extracted revision string
namedFieldsShare float Share (0.0–1.0) of terminal field names containing a readable 4+ letter word token
verdict enum xfa_hybrid (114), acroform (15), flat (5), xfa_dynamic (0), scanned (0)
note string Collection anomalies (e.g., landing page 404 while PDF path remained live, empty /AcroForm dicts)

Summary Statistics from the Snapshot:

  • 85% (114 / 134) are xfa_hybrid: 67/68 IRS forms, 24/26 USCIS forms, 10/13 California court forms, 4/4 SSA forms, and 5/5 VA forms include an Adobe /XFA layer alongside AcroForm widgets.
  • 87% (116 / 134) were produced with Adobe LiveCycle Designer or Adobe Designer: Despite Adobe sunsetting LiveCycle years ago, Designer 6.5 and Adobe LiveCycle Designer ES 8.2 / 9.0 / 10.0 dominate federal and state forms.
  • 49% (63 / 129) of fillable forms have opaque field names (<10% readable): 61 of 68 IRS forms have namedFieldsShare < 0.10 (most are 0.0, naming every field f1_1[0], f1_2[0], etc.), whereas USCIS, SSA, and VA use semantic camelCase names (LastName, ClaimantSSN).
  • 34% (45 / 134) carry owner encryption: All 45 permit form filling, 44 deny document modification (modifyDenied), and 16 deny annotations (annotateDenied).
  • Outliers by field count: IRS Form 709 leads with 1,851 fields across 10 pages, followed by USCIS Form I-129 (980 fields, 38 pages), USCIS Form I-485 (760 fields, 24 pages), and IRS Form W-2 (568 fields, 11 pages).

Bonus Dataset: Real-World AI PDF Field Detector Correction Telemetry

Alongside the structural government form study, we also published a reproducible 5-row PDF mail-merge benchmark fixture and real user correction logs (5,674 detected fields across 43 flat/scanned documents: 2.2% false-positive deletions, 1.2% missed-field additions, 3.5% geometry adjustments):
https://justfill.app/research/ai-pdf-form-filling-reliability

License: CC BY 4.0 (Attribution: JustFill, 2026-09-02). Feel free to use the CSV for visualizations, PDF parser testing, or civic-tech research!


r/datasets • • 13h ago

question Data Bounty Hunter Seeks Data Buyers

1 Upvotes

Researchers and Trainers, what data do you really really want, heck, would pay for? What are you paying for now, and not getting….

What if I told you I am a data bounty hunter who will go out and find it for you?


r/datasets • • 18h ago

request Looking for Knife/Weapons/Guns Dataset

1 Upvotes

Hello can you guys give me links to datasets of guns, knife, or other weapons