From news bursts to story graphs, with local LLMs

This post traces how tweets or news stories spread through the media and whether we can visualise the spread of news or items visually. Typically, items start with small bursts and then become full-blown stories as more people connect and discuss it online.

The plan was simple to state:

posts → people, places, organisations and claims → a graph that changes over time → “what’s happening?”

Originally, I wanted to use X f(ormerly Twitter) and listen to tweets live, but the cost of the X stream is prohibitive, so I switched to Bluesky whose whole public stream is free to read.

The the idea was to identify news/event bursts and then see how the event or news item actually becomes a full-blown story. The interesting part is what happens over the next day. So the project was created in two halves 1) finding bursts, and 2) following each burst into a story.

The setup

• Data: Bluesky’s public stream through its free Jetstream feed, English posts only. This was done for four days from 18 to 22 October 2026, and then frozen so every number below comes from the same recording: about 8 million original posts and 7 million reposts.
• Everything runs offline on the recording. Recording happens once; every experiment replays the saved files. That made it possible to change a threshold and compare results on identical data.

Job	Model (all local)	Why this one
Find names in every post	GLiNER, a small entity model	fast enough for every post, about 140 posts a second
Judge whether posts say the same thing	MiniLM sentence embeddings	about 7,000 posts a second
Extract facts and claims for a story	GLM-4.7-Flash, via Ollama	the fastest model
Tamil news, merging names, captions	Gemma 4 31B, via Ollama	best quality , and it reads Tamil
Store the graph	Neo4j Community edition	free and local

How it works

1. Record the Bluesky stream to hourly files.
2. Find the names in each post: people, organisations, places, facilities, products and events.
3. Detect bursts: a set of names that suddenly many more people are writing about than usual.
4. Follow the story: collect its posts from 3 hours before the burst to 24 hours after.
5. Extract facts and claims at 12 points in time with a local model, each tied to the posts that support it.
6. Merge duplicate names, store everything in Neo4j, and draw the story as an interactive page that plays the 12 steps in order.

Part 1: Bursts, or finding the moment a story breaks

1. Counting hashtags finds bots

The obvious first step is to count hashtags and words and flag whatever grows fastest. In the first morning of data, the top trends were:

Trend	Posts	Authors
roadworks (a UK traffic bot)	564	1
#kickban	197	1
#greatwhitemike	158	1
nebraska	43	32

Three of the top four were single accounts. Everything from here on counts distinct people, not posts, and nearly identical texts count once, because spam networks post the same template from dozens of accounts (one sent the same advert from 82).

2. Names, not hashtags

Only 11% of Bluesky posts carry a hashtag, names were needed inside the text. Gliner and Spacy entity extractors were compared on 200 posts, using Gemma 4 31B as the reference. Gemma is too slow for every post, at 0.3 posts a second, but it is accurate.

Extractor	Precision	Recall	Types right	Speed
spaCy (small)	0.77	0.67	0.59	about 460 posts a second
GLiNER	0.48	0.81	0.72	about 160 posts a second
GLiNER with a junk filter	0.73	0.72	0.75	about 160 posts a second

Unfiltered, GLiNER labelled “I’m 49”, “Dinner” and “He” as entities. A small filter (no pronouns, no dates, no common phrases in lower case) fixed most of it. Over a full hour, 53% of posts named nothing, 24% named one thing, and only 23% named two or more. Short posts mostly mention one part of a story.

3. One story, seven events

The initial detector linked names that people mention together. On the morning of 18 October, Ukrainian drones struck targets in St Petersburg: an oil terminal, a VK data centre, an Ozon warehouse and a bridge. The detector reported seven separate events. One post said “Ozon”, another “VK data center”, a third “oil terminal”, and almost nobody named two of them in one post.

4. Need for embeddings

Names alone could not tell which posts belonged to the same story, because most posts name only one part of it. I needed a way to compare what posts say, not just which names they share.

An embedding turns a post into a list of numbers, so that posts with similar meaning get similar numbers even when they share no words. “Drones hit an oil terminal in St Petersburg” and “Ozon warehouse on fire after the attack” come out close together; a post about a football score does not. A small local model (MiniLM) does this at about 7,000 posts a second, so it can be applied freely. Reading every post with a language model would understand meaning better, but Gemma manages 0.3 posts a second while 16 to 44 arrive every second.

I ended up using embeddings in four places:

Where	What went wrong with names alone	What embeddings did
Grouping a burst into one event	The strike came out as 7 events, because “Ozon”, “VK data center” and “oil terminal” were rarely named together	Names whose posts read alike were linked, and the strike became one event with 8 names
Following an event from one window to the next	When later posts used new names (“Baltic oil terminal”, “Saint Petersburg”), the strike restarted as a new event	The new group was matched to the existing event by meaning, and the strike stayed one event for 105 minutes
Choosing the posts to read for an event	A post that only said “Nintendo” or “Korea” was pulled into unrelated stories: a 22 year old game, a Korean sports team	A post sharing only one name with the event must also read like the event
Following a story for a day	Every passing mention of a name would have joined the story	Posts naming one of the story’s names must also be close in meaning to the story’s own posts

Two limits showed up. The threshold matters: at a similarity of 0.45, unrelated political stories merged, because political posts all read alike to a small model; at 0.65 the strike split again; 0.55 worked. And embeddings cannot rescue a story whose names are too general, which comes up in Part 2.

Each change was tested on the same three hours of data:

Run	Change	Events found	What happened
1	names mentioned together	173	the strike came out as 7 events, a college football game as 7
2	linking posts that read alike	35	the strike disappeared altogether (see the next section)
4	writers counted per event	60	the strike became one event with 8 names
7	thresholds tuned	61	the strike lasted 105 minutes as one event, and unrelated politics stopped merging

5. The burst formula

Putting the pieces together, this is how a burst is scored. Every 5 minutes, for every name, the detector looks at the last 15 minutes:

• authors: the distinct people who wrote or reposted a post naming it
• total: the distinct people active at all in those 15 minutes
• share: authors divided by total

Each name has its own baseline: its usual share, and how much that share normally varies. Both are running averages that remember roughly the last 6 hours (a half life of 72 steps of 5 minutes):

Each part has a job:

• Share, not count. Activity on Bluesky roughly triples between the US night and morning. Counts would make every name look like a burst each morning; shares only move when the conversation does.
• The expected term allows for ordinary chance: a name expected 100 times can easily turn up 110 times.
• The (total × √var)² term asks more of names that are always busy and always vary, such as Trump.
• The + 1 stops a name never seen before from scoring infinity with two people, while a new name with 20 people still scores about 20.
• The baseline is updated after scoring, so a burst is never compared with itself.

A name is bursting when at least 8 people named it in the window. Bursting names are grouped into events (by co-mention and by embeddings), and an event must then pass two more tests from the previous section: at least 3 people wrote original posts about it, and for at least one of its names the writers alone score at least 2 on a second baseline that counts writers only. The first 2 hours of any run only build the baselines.

Two examples. At 06:45 IST on 18 October, Ozon was named by 21 people in 15 minutes where about 0.8 would be normal, giving z = 7.6, and VK data center by 11 people where almost none was expected, giving z = 9.1: both bursts. For a constantly busy name, take an illustration: a usual share of 3% of 15,000 active people gives an expected 450; if its share normally varies by 0.4 points, the spread term adds 60, so the denominator is about 64. Then 520 people give z = 1.1, which is ordinary, and it would take about 770 to reach z = 5.

7. The burst is short

Every tech event the detector found on Monday 19 October lasted between 10 and 105 minutes. That is when people suddenly pile in. But the stories themselves ran for a day:

The shaded band is all the detector saw. For the strike, the biggest wave came six hours later, in the Indian evening, as casualty figures and claims arrived. Prop 40 had a second wave the next evening. A graph built from the burst alone had 2 to 5 checkpoints: a snapshot, not a story.

Part 2: From a burst to a story

1. Following a story for a day

So the burst became a seed. Each detected event gives its main names and the average embedding of its posts. One pass over the recording, from 3 hours before the burst to 24 hours after, collects every post that names two of the story’s names, or names one and reads like the story, along with its repost count. All stories are followed in the same pass:

Story	Posts followed	Reposts of them
Steam Next Fest	2,312	7,763
Nvidia and warehoused GPUs	794	3,398
St Petersburg drone strike	414	3,784
Prop 40 and Sergey Brin	342	4,044
Amazon GPUs	286	2,965

2. Twelve checkpoints, facts and claims

The followed posts are cut into 12 slices with similar numbers of posts, so busy periods get finer steps. For each slice, GLM-4.7-Flash reads the most reposted posts (and the first one) and returns:

• Facts as subject → verb → object (“Ukrainian drones → attacked → Ozon warehouse”), each citing the posts that state it. A fact citing no real post is dropped.
• Claims, with who makes them: “135 drones were shot down” (the regional authorities), “our services are working normally” (VK), “the warehouse was used by the Russian defence ministry” (Ukraine; Ozon denies it).

Separating facts from attributed claims turned out to be one of the most useful parts. A story is mostly claims: casualty figures from three different sources, accusations, denials.

3. Many names for one thing

Extraction runs slice by slice, and the same thing comes back under different names: “Vijay”, “CM Vijay”, “C Joseph Vijay” and “TN CM”; “IMD”, “India Meteorological Department” and “weather department”. Three layers fixed most of it:

1. Prevention: each slice is given the names already used for this story, with an instruction to reuse them, plus a short list of preferred verbs (“attacked”, “located in”, “deployed to” and so on).
2. Clean up in code: “X says Y” becomes a claim by X, not a fact; a fact linking a thing to itself is dropped; “VK data centre was hit by a drone” is turned round to “drone hit VK data centre”.
3. One Gemma call per story groups the names that mean the same thing and picks one, while being told that a district is not its city (North Chennai is not Chennai).

On the Chennai rain story (below), this cut 127 different names to 70, and 42 different verbs to 11. Gemma also knew that “Kovai” is Coimbatore and that “the Tamil Nadu Weatherman” is the blogger Pradeep John. It made mistakes too: it merged “weather centre”, which in Tamil reports usually means Chennai’s regional centre, into the national IMD.

4. Captions that say only what’s new

My first pages showed each slice’s summary, and every summary began by restating the whole story. So Gemma now writes one fixed sentence for the story, and one sentence per step with only what is new. For Prop 40, California’s proposed tax on billionaires:

Story: Users debate California’s proposed billionaire wealth tax (Prop 40) and criticise tech billionaires’ ties to Donald Trump and Mar-a-Lago.

1. Users suggest renaming Mar-a-Lago and discuss Props 41 and 42.
2. Sergey Brin reportedly spent over $100 million against Prop 40 and dined with Trump at Mar-a-Lago.
3. Reports that Mark Zuckerberg moved to Florida and Gavin Newsom vacationed with Brin.
4. Musk, Thiel and Zuckerberg reportedly oppose Prop 40; Bernie Sanders supports it.
5. A poll suggests Prop 40 faces a “spectacular collapse”.
6. Brin reportedly moved to Nevada and bought a Miami home; the BBC reportedly demanded Mar-a-Lago’s membership list.
7. Economist Gabriel Zucman explains Prop 40 as a wealth tax; Mark Ruffalo warns of health insurance losses.

And for the strike, the account firms up over the day: targets first, then the radar site and bridge, then Ozon’s denial, then casualty figures from three different sources (Zelensky, the Russian defence ministry and the city’s mayor, each with different numbers), then “over 1,000 drones, the largest attack on the city”. These are what posts claimed, not checked facts, which is why every claim keeps its source.

The pages play the 12 steps in order, with a speed control (2 to 10 seconds per step) and each step’s caption over the graph. They work on a phone.

5. When following goes wrong

The Nvidia story shows the weakness. Its seed names were “Nvidia”, “GPUs”, “RAM” and “Jensen Huang”, all of which people discuss constantly. So the day’s story became everything about Nvidia: warehoused GPUs, “channel stuffing” accusations, a German AI cloud deal, a new CPU. Every Nvidia post read like the story, so embeddings could not hold it together either. You can see it in the figure above: a flat, steady line rather than a wave. A story can only be followed if its names are specific to it. Steam Next Fest, a festival of hundreds of game demos, had the opposite problem: a few prolific developers filled the evidence with their own game.

A detour to Chennai: a story told in two languages

Heavy northeast monsoon rain hit Chennai on 17 and 18 October while I was recording. Bluesky had almost nothing about it (128 posts in 21 hours, mostly automatic copies of Tamil TV headlines), so I followed 1,621 news articles instead, in English and Tamil, alongside Open-Meteo’s hourly rainfall.

• GLM cannot read Tamil. It returned nothing for seven Tamil headlines. Gemma 4 31B read them and wrote every name in English (சென்னை → Chennai, தாம்பரம் → Tambaram), so Tamil and English articles about the same place joined up in the graph.
• News arrived the morning after. The heaviest rain fell around 00:30 IST; articles peaked six to nine hours later, and Tamil outlets were out first, from 06:30, with English coverage peaking around midday.

• The figures firmed up through the day: 175 mm at Parrys, then “30 cm in Chennai”, then 36.34 cm overnight. The official and critical accounts sat side by side: the Corporation’s “no fatalities” and “drained by dawn, barring Perambur and Velachery”, a minister’s “Chennai cannot withstand 24 hours of continuous rain”, and a resident’s “36 cm fell in 3 hours and the weather centre gave no warning”.

One caveat: the rainfall model under reads badly. It peaked at 16 mm an hour, while the reported totals were ten times higher. It gets the timing right, not the amount.

Play the stories

Each story is an interactive page: press Play to watch it build up step by step, with a caption saying what changed, and use the slider to jump to any step. They work on a phone too. All eight are listed at https://tvganesh.github.io/bluesky-stories/.

• St Petersburg drone strike: targets, denials and casualty figures from three sources, over a day.
• Prop 40 and Sergey Brin: California’s billionaire tax, Brin’s opposition and his reported move.
• Chennai monsoon rain: the story from English and Tamil news, alert by alert.
• Nvidia and warehoused GPUs: the story that drifted into everything about Nvidia.

The pages show what the models extracted, not the posts themselves: account names, links to posts and quoted posts have been removed, and posts their authors deleted after I recorded them were dropped.

This was an interesting exercise to see how the stories develop. What would be more interesting is so what cause tweets or stories to go viral. That is definitely food for thought.

Also see

• Building a Flight Simulator with Claude Code, Opus 5
• Building an Agentic Stock Picker with Anthropic
• IPL 2023:GooglyPlusPlus now with by AI/ML models, near real-time analytics!
• Analyzing player performance with animated charts!
• Using Reinforcement Learning to solve Gridworld

To see all posts click Index of posts