Twitter Sentiment Analysis in Python (2026)

Brand teams, product owners, election desks, and crypto traders all want the same thing from Twitter data: a number that says whether the mood around a topic is moving up or down. Two things historically got in the way. Pulling enough tweets used to be expensive, and the off-the-shelf sentiment models were trained on movie reviews, so they mislabeled half of real social copy. This walkthrough closes both gaps. You collect tweets through the advanced search endpoint at $0.0008 per page, then score them with whichever of three engines fits your speed-versus-accuracy budget, and finish with a two-panel matplotlib chart. Every block runs on Python 3.11 or newer, and nothing in the fast path needs a GPU.

Sentiment analysis is one of the most common reasons people reach for tweet data in the first place, alongside trend tracking and competitor monitoring. If you are new to the data side, the broader Python Twitter API tutorial walks through the same collection layer in more general terms, and the complete Twitter API tutorial for 2026 covers the read and write surface end to end. This post stays narrow on purpose: get clean tweets in, get a defensible sentiment score out, and keep the bill in the single-digit-cents range while you experiment.

The framing here matches how practitioners describe the work in public. When people lay out the menu of things you can do with tweet data, sentiment analysis is almost always the first lens they reach for, the positive-negative-neutral read around a topic or brand, ahead of volume counts or network mapping.

https://x.com/Sriiii91/status/1843811781791064183

Here is the money math up front so there are no surprises. The search endpoint returns about 20 tweets per page, one billable call each. Ten thousand tweets is therefore roughly 500 calls, which at $0.0008 a call is $0.40. In per-volume terms that is $0.04 per 1,000 tweets. Your account starts with $0.50 of free credit, around 625 calls or 12,500 tweets, so the entire tutorial runs before you ever attach a card. That pricing is the whole reason this kind of project is now a weekend build rather than a budget line: for a side-by-side of what reads actually cost across the market, the Twitter API cost breakdown and the per-1,000-tweet cost benchmark put the numbers next to the alternatives. If you are still deciding whether to pay at all, the rundown on whether the Twitter API is free in 2026 is worth five minutes before you write a line of code.

---

What you will build

The pipeline has five moving parts, and each one is a small, swappable function:

1. A collector that walks the search cursor and returns raw tweet dicts.
2. A cleaner that strips links and handles so the scorer sees real words.
3. Three scoring engines (lexicon, rule-based, neural) you can switch between without touching the rest of the code.
4. A rollup that turns thousands of labeled rows into a daily mean and a volume count.
5. A chart that puts sentiment and volume on one figure so spikes are obvious.

Keep the layers decoupled and you can later trade VADER for a transformer, or swap a CSV sink for Postgres, without a rewrite. The decoupling is not academic. In practice you will start with VADER because it is instant, ship a first chart, then discover your topic is sarcasm-heavy and quietly swap in the transformer at the scoring layer while everything upstream and downstream stays untouched. That is the payoff of treating each stage as a pure function: the collector does not care how you score, the scorer does not care how you collected, and the chart does not care which engine produced the labels. If you have built a collection pipeline before for a different purpose, say a Twitter bot or a one-off tweet scrape, the collector here will feel familiar, just pointed at search instead of a timeline.

---

Setting up the environment

pip install requests textblob vaderSentiment transformers torch matplotlib pandas python-dotenv
python -m textblob.download_corpora

Drop your key into a .env file so it never lands in source control:

TWITTERAPIS_KEY=your_key_here

Grab the key at twitterapis.com. Signup takes under a minute, asks for no developer application, no phone number, and no card, and credits your account with $0.50 automatically. If you have wrestled with the official console before, the contrast is stark: the standard route still involves a developer agreement, app registration, and project setup, which the step-by-step on how to get a Twitter API key lays out so you can see exactly what you are skipping. The one habit worth keeping from the official flow is treating the key like a password. Load it from the environment, never paste it into a notebook cell you might share, and rotate it from the dashboard if it ever leaks into a screenshot or a git commit.

---

Step 1: Harvest tweets from advanced search

The advanced search endpoint takes the full X operator syntax and pages through results with a cursor. Each page is about 20 tweets and costs $0.0008, so the code tracks pages to keep the running cost honest.

import os
import time
import requests
from dotenv import load_dotenv

load_dotenv()

KEY = os.environ["TWITTERAPIS_KEY"]
ROOT = "https://api.twitterapis.com"
AUTH = {"Authorization": f"Bearer {KEY}"}

PER_PAGE = 20          # advanced_search returns ~20 tweets per page
PAGE_PRICE = 0.0008    # one page == one billable read call


def harvest_tweets(
    query: str,
    page_limit: int = 25,
    start_time: str | None = None,
    end_time: str | None = None,
) -> list[dict]:
    """
    Walk the search cursor for `query` and return raw tweet dicts.
    Stops at page_limit pages. Roughly PER_PAGE tweets land per page,
    and every page bills PAGE_PRICE.
    """
    endpoint = f"{ROOT}/twitter/tweet/advanced_search"
    args: dict = {"query": query, "product": "Latest"}
    if start_time:
        args["start_time"] = start_time
    if end_time:
        args["end_time"] = end_time

    collected: list[dict] = []
    cursor: str | None = None
    pages = 0

    while pages < page_limit:
        if cursor:
            args["cursor"] = cursor
        elif "cursor" in args:
            del args["cursor"]

        r = requests.get(endpoint, headers=AUTH, params=args, timeout=15)
        r.raise_for_status()
        payload = r.json()

        batch = payload.get("tweets", [])
        if not batch:
            break
        collected.extend(batch)
        pages += 1

        cursor = payload.get("next_cursor")
        if not cursor:
            break

        time.sleep(0.05)   # courtesy gap between pages

    return collected


def spend(tweet_count: int) -> float:
    return tweet_count / PER_PAGE * PAGE_PRICE


sample = harvest_tweets(
    query='("AI tools" OR "AI app") lang:en -is:retweet',
    page_limit=25,
)
print(f"Pulled {len(sample)} tweets | est. spend ${spend(len(sample)):.4f}")

Twenty-five pages is roughly 500 tweets for an estimated $0.02. Authentication is a single bearer header, so there is no OAuth handshake, no token refresh, and no four-credential dance.

Two design choices in that function are worth calling out, because they are where collectors quietly go wrong. First, product is set to Latest, which walks the timeline newest to oldest. That ordering matters for a daily rollup: if you stop early you lose the oldest tweets in your window, not a random slice, so an under-pulled chart will show a clean recent trend with a missing tail rather than evenly thinned data. Second, the cursor is the only source of truth for "is there more". A non-empty tweets array with no next_cursor means you reached the end of what matches, and the loop should stop, which it does. The most common bug people hit is reusing a stale cursor across two different queries, which silently returns the wrong page. Keep one cursor per query and reset it whenever the query string changes. The full grammar for that query string, every operator you can stack, lives in the advanced search operators reference, and the X documentation itself spells out the canonical search query syntax if you want the upstream source.

For deeper collection patterns beyond this one function, the writeups on the best Twitter API for scraping and on choosing the cheapest Twitter API in 2026 compare the collection layer across providers, and the how to scrape tweets guide covers the timeline and user-tweets variants of the same loop.

A note on writes before we move on. Sentiment work only reads tweets, but the same key also reaches the 12 write actions, the simple toggles (favorite and unfavorite, retweet and unretweet, bookmark and unbookmark, follow and unfollow, plus delete) and media upload at $0.0008 per call, with tweet creation and DM send at $0.0016, in case you later want to act on a sentiment trigger. Those writes take a bring-your-own auth_token and ct0 on each request, and the service never stores them. There are 48 endpoints in total, 34 read and 14 write. Closing that loop, scoring sentiment and then favoriting or replying to the tweets that cross a line, is exactly the kind of automation the Twitter bot build guide walks through, and it is why keeping the read and write surfaces under one key is convenient: you never have to re-authenticate to act on what you just measured.

---

Step 2: Clean the text before scoring

Raw tweets are full of URLs, @handles, and stray whitespace. Lexicon models score those tokens as neutral noise and a transformer wastes attention on them, so a quick cleaner pays off across every engine.

import re

_URL = re.compile(r"https?://\S+")
_MENTION = re.compile(r"@\w+")
_SPACES = re.compile(r"\s+")


def tidy(text: str) -> str:
    text = _URL.sub("", text)
    text = _MENTION.sub("", text)
    return _SPACES.sub(" ", text).strip()

Keep hashtags. The word inside a hashtag often carries the sentiment ("#disappointed"), and all three engines read it as a normal token once the pound sign is attached to real text.

Resist the urge to clean more aggressively than this. Two over-cleaning mistakes are common and both cost you accuracy. Lowercasing everything destroys the ALL-CAPS signal that VADER specifically reads as intensity, so a furious "THIS IS BROKEN" collapses into a calm "this is broken". Stripping punctuation flattens the exclamation runs and question marks that the VADER lexicon was built to weigh. The cleaner above does the minimum that helps every engine, removing URLs and handles that carry no sentiment, and leaves the rest alone. There is one more edge case worth a guard: a tweet that is nothing but a link and a mention cleans down to an empty string, and an empty string scores as neutral by default. The transformer step later falls back to the raw text when cleaning empties it, which is the correct behavior, because a neutral label invented from an empty string is worse than a slightly noisy label from the original.

---

Step 3: Three scoring engines, fastest to most accurate

There is no single right model. The choice is a budget decision between throughput and label quality. We will wire up all three and let you pick per job.

3a: TextBlob, the throwaway baseline

TextBlob returns a polarity float from -1 to +1. Its lexicon, documented in the TextBlob reference, came mostly from product reviews, so it is shaky on slang and emoji, but it installs in seconds and is a useful sanity check.

from textblob import TextBlob


def tb_polarity(text: str) -> tuple[str, float]:
    polarity = TextBlob(text).sentiment.polarity
    if polarity > 0.1:
        return "positive", polarity
    if polarity < -0.1:
        return "negative", polarity
    return "neutral", polarity


for tw in sample[:5]:
    label, score = tb_polarity(tidy(tw["text"]))
    print(f"{label:8s} {score:+.2f}  {tw['text'][:70]}")

Treat TextBlob as a smoke test. If even TextBlob and VADER disagree wildly on your corpus, that is a signal your topic is sarcasm-heavy and you should jump straight to the transformer.

3b: VADER, tuned for social text

VADER (Valence Aware Dictionary and sEntiment Reasoner) was designed for short social posts. It understands ALL CAPS, repeated punctuation, intensifiers, negation, and emoji, runs in microseconds, and needs no GPU. The compound score is the summary number, and the standard cutoffs come from the original paper (Hutto and Gilbert, 2014, ICWSM): at or above 0.05 is positive, at or below -0.05 is negative, the band between is neutral.

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

_vader = SentimentIntensityAnalyzer()


def vader_compound(text: str) -> tuple[str, float]:
    compound = _vader.polarity_scores(text)["compound"]
    if compound >= 0.05:
        return "positive", compound
    if compound <= -0.05:
        return "negative", compound
    return "neutral", compound


for tw in sample[:5]:
    label, score = vader_compound(tw["text"])   # VADER likes the emoji, so skip tidy()
    print(f"{label:8s} {score:+.2f}  {tw['text'][:70]}")

One subtlety: feed VADER the raw tweet, not the cleaned one, because the emoji and casing it reads are exactly the signal you stripped out in Step 2. For broad trend lines VADER is dependable. It stumbles on irony and niche jargon, which is the transformer's cue.

It helps to understand why VADER is so fast and so well-suited here. It is not a model in the machine-learning sense, it is a hand-built lexicon plus a handful of grammatical heuristics, so there is no inference step and no warm-up cost. The reference implementation and the original paper (Hutto and Gilbert, 2014, ICWSM) lay out exactly which rules fire: a word like "good" gets a base valence, "very good" amplifies it, "not good" flips it, and an exclamation point nudges the magnitude. Because every rule is explicit, you can read the lexicon and predict failures. It has no entry for a brand-new slang term or an ironic "great, another outage", so it will score those wrong with high confidence. That predictability is a feature when you are debugging a strange aggregate: you can almost always trace a surprising VADER label back to a specific token in the text.

3c: A tweet-tuned transformer, when labels must be right

The cardiffnlp/twitter-roberta-base-sentiment model was fine-tuned on tens of millions of tweets and predicts three classes: negative (LABEL_0), neutral (LABEL_1), positive (LABEL_2). On CPU budget 5 to 50 ms per tweet depending on length; on a GPU it drops near 2 ms. The class split it reports tracks the SemEval-2017 Task 4 tweet-sentiment benchmark it was evaluated against, which is the most-cited shared task for exactly this problem and a useful anchor when someone asks "how good is good enough".

from transformers import pipeline

# First call downloads ~500 MB, then loads from local cache.
_clf = pipeline(
    "sentiment-analysis",
    model="cardiffnlp/twitter-roberta-base-sentiment",
    truncation=True,
    max_length=128,
)

_NAME = {"LABEL_0": "negative", "LABEL_1": "neutral", "LABEL_2": "positive"}


def roberta_batch(texts: list[str]) -> list[tuple[str, float]]:
    cleaned = [tidy(t) or t for t in texts]   # fall back to raw if cleaning empties it
    out = _clf(cleaned)
    return [(_NAME.get(r["label"], r["label"]), round(r["score"], 4)) for r in out]


pairs = roberta_batch([tw["text"] for tw in sample[:5]])
for tw, (label, conf) in zip(sample[:5], pairs):
    print(f"{label:8s} {conf:.2f}  {tw['text'][:70]}")

Batching matters. Passing a list lets the transformers pipeline pack sequences together, which is several times faster than calling it once per tweet. On a recent laptop CPU, 500 tweets finish in roughly 30 to 60 seconds; on a free-tier Colab T4 GPU, under 10 seconds. Two knobs in the pipeline call do real work and are easy to miss. truncation=True with max_length=128 keeps a single long tweet, or a thread you accidentally concatenated, from blowing up memory or stalling on a 2,000-token input; 128 tokens comfortably covers a 280-character tweet with room to spare. And the first call quietly downloads roughly 500 MB of weights, so in a serverless or container context you want that download to happen at build time, not on the first user request, or your cold start will look like a hang. Warm the model once at import and the per-tweet cost is all that remains.

The NLTK-plus-transformers stack shows up constantly in public tutorials, which is a good sign you are on a well-trodden path rather than an exotic one when you trade a lexicon for a fine-tuned model. The same building blocks generalize past tweets, to product reviews and any other short-text corpus.

https://x.com/PythonPr/status/2052051331402367444

3d: Scoring tweets that are not in English

The moment you drop lang:en from a query, English-only models start guessing. Swap in the multilingual sibling, cardiffnlp/twitter-xlm-roberta-base-sentiment, which covers roughly eight languages including Spanish, French, German, Portuguese, Italian, and Arabic, and slots into the exact same pipeline call.

_multi = pipeline(
    "sentiment-analysis",
    model="cardiffnlp/twitter-xlm-roberta-base-sentiment",
    truncation=True,
    max_length=128,
)


def multilingual_label(text: str) -> tuple[str, float]:
    r = _multi(tidy(text) or text)[0]
    return r["label"].lower(), round(r["score"], 4)

The architecture and label set match the English model, so everything downstream (the DataFrame build, the rollup, the chart) is unchanged. For long-tail languages outside that set, a lighter distilled multilingual classifier covers more of them at a small accuracy cost. The practical rule: detect the dominant language of your sample first, and only reach for the multilingual model when a meaningful share of tweets is not English, because the English-only model is a touch sharper on English text.

A subtle trap with multilingual sentiment is mixed-language tweets, the kind that splice English brand names and hashtags into a Spanish or Arabic sentence. The XLM model handles those far better than the English-only one, but you will still see lower confidence scores, because the model is genuinely less certain. Do not treat that lower confidence as a defect to clean away. Instead, route low-confidence rows to a human spot-check or simply weight them less in the aggregate. The cleanest approach for a multi-region brand is to bucket by lang at the query level, run each language through the model best suited to it, and only then merge the labeled rows into one frame. That keeps the per-language accuracy high and makes a regional breakdown trivial later, since the language tag is already on every row.

---

Step 4: Assemble a labeled DataFrame

With the engines defined, annotate the whole pull and load it into pandas. We carry both the VADER and transformer labels so you can audit where they diverge later.

import pandas as pd

texts = [tw.get("text", "") for tw in sample]
roberta_pairs = roberta_batch(texts)

rows = []
for tw, (rb_label, rb_conf) in zip(sample, roberta_pairs):
    text = tw.get("text", "")
    v_label, v_score = vader_compound(text)
    metrics = tw.get("public_metrics", {})
    rows.append(
        {
            "id": tw.get("id"),
            "created_at": tw.get("created_at"),
            "text": text,
            "likes": metrics.get("like_count", tw.get("favorite_count", 0)),
            "retweets": metrics.get("retweet_count", tw.get("retweet_count", 0)),
            "vader": v_label,
            "vader_score": v_score,
            "roberta": rb_label,
            "roberta_conf": rb_conf,
        }
    )

frame = pd.DataFrame(rows)
frame["created_at"] = pd.to_datetime(frame["created_at"])
frame["day"] = frame["created_at"].dt.date

print(frame[["day", "vader", "roberta", "roberta_conf"]].head(10))
print("\nTransformer class split:")
print(frame["roberta"].value_counts())

agree = (frame["vader"] == frame["roberta"]).mean()
print(f"\nVADER and RoBERTa agree on {agree:.0%} of tweets")

That agreement rate is a free diagnostic. When the two engines line up on 80 percent or more of your sample, VADER alone is good enough and you can drop the transformer to save time. When they diverge a lot, your topic is hard and the transformer labels are the ones to trust.

Carrying both labels in the same frame costs almost nothing and pays off in three ways. It gives you the agreement metric above for free. It lets you reconstruct a decision later, since you can always see what each engine thought about any given tweet without re-running anything. And it sets up the production routing pattern from later in this post, where VADER handles the bulk and the transformer is reserved for the rows where the two disagree or VADER sits near its neutral boundary. One detail to get right while building the frame: parse created_at into a real datetime immediately, as the code does, because every time-series operation downstream, the daily grouping, the rolling mean, the chart axis, depends on pandas understanding these as timestamps rather than strings. The pandas groupby reference covers the aggregation you will lean on in the next step, and getting the dtype right here is what makes that one-liner work.

---

Step 5: Roll up by day and chart

A column of labels is not a finding. Convert labels to a numeric score, average per day, and plot the trend against volume so a sentiment dip that coincides with a volume spike jumps off the page.

import matplotlib.pyplot as plt
import matplotlib.dates as mdates

POLARITY = {"positive": 1, "neutral": 0, "negative": -1}
frame["polarity"] = frame["roberta"].map(POLARITY)

daily_rollup = (
    frame.groupby("day")
    .agg(mean_polarity=("polarity", "mean"), volume=("id", "count"))
    .reset_index()
)
daily_rollup["day"] = pd.to_datetime(daily_rollup["day"])

fig, (top, bottom) = plt.subplots(2, 1, figsize=(12, 7), sharex=True)

top.plot(daily_rollup["day"], daily_rollup["mean_polarity"],
         color="#16a34a", linewidth=2, marker="o", markersize=4)
top.axhline(0, color="#666", linewidth=0.8, linestyle="--")
top.set_ylabel("Mean polarity")
top.set_title('Daily sentiment for the "AI tools" query', fontsize=13)
top.set_ylim(-1.1, 1.1)
top.fill_between(daily_rollup["day"], daily_rollup["mean_polarity"], 0,
                 where=daily_rollup["mean_polarity"] >= 0, alpha=0.15, color="#16a34a")
top.fill_between(daily_rollup["day"], daily_rollup["mean_polarity"], 0,
                 where=daily_rollup["mean_polarity"] < 0, alpha=0.15, color="#dc2626")

bottom.bar(daily_rollup["day"], daily_rollup["volume"],
           color="#2563eb", alpha=0.7, width=0.6)
bottom.set_ylabel("Tweet volume")
bottom.xaxis.set_major_formatter(mdates.DateFormatter("%b %d"))
fig.autofmt_xdate()

plt.tight_layout()
plt.savefig("sentiment_trend.png", dpi=150)
print("Saved sentiment_trend.png")

Read the two panels together. Rising volume with falling polarity is the classic shape of an outage, a recall, or a bad news cycle. Rising volume with rising polarity usually means a launch landed well. Flat volume with drifting polarity is slow opinion change, the kind worth a weekly digest rather than an alert.

The two-panel layout is doing deliberate work, and the matplotlib subplots call with sharex=True is what makes it readable: both panels share one time axis, so a spike in the bottom panel lines up vertically with the polarity reading directly above it. Without the shared axis your eye has to do the alignment manually, and that is where misreadings creep in. A few presentation choices matter more than they look. Filling positive area green and negative area red gives a stakeholder the gist in half a second before they read a single number. Pinning the y-axis to the full minus-one-to-one range, rather than letting matplotlib auto-scale, stops a quiet week from looking like a crisis just because the software zoomed into a tiny band of variation. And plotting raw volume alongside sentiment is the single most important habit here, because a sentiment number with no volume context is meaningless: minus-0.8 across nine tweets is noise, minus-0.4 across nine thousand is a five-alarm fire, and only the volume panel tells you which one you are looking at.

---

Step 6: Weight sentiment by engagement

Counting tweets equally hides the fact that a few highly-liked posts shape the conversation. Weighting by likes often flips the headline number, so it is worth a dedicated pass when you report to a stakeholder.

def weighted_positivity(df: pd.DataFrame) -> float:
    weight = df["likes"].clip(lower=1)
    positive_weight = weight[df["roberta"] == "positive"].sum()
    return positive_weight / weight.sum() * 100


raw_pos = (frame["roberta"] == "positive").mean() * 100
print(f"Raw positive share:        {raw_pos:.1f}%")
print(f"Engagement-weighted share: {weighted_positivity(frame):.1f}%")

If the engagement-weighted number lands well below the raw share, a small cluster of popular complaints is dominating, and that is exactly the signal a brand team needs before it shows up in the press.

The clip(lower=1) in that function is not cosmetic. Without it, every zero-like tweet contributes zero weight and effectively disappears from the denominator, which quietly inflates the influence of the handful of viral posts and makes the weighted number swing wildly run to run. The floor of one keeps quiet tweets in the conversation while still letting popular ones pull harder. Likes are the simplest weight, but they are not the only sensible one. Retweets capture spread rather than approval, which is a different and sometimes more important signal for a crisis, since an angry tweet that gets retweeted ten thousand times is reaching far more people than one that merely gets liked. A reasonable composite is likes plus a multiple of retweets, tuned to how much you care about reach versus endorsement. Whatever you choose, document it next to the number, because a weighted percentage with no stated weighting scheme is impossible for a stakeholder to interpret or compare across reports.

Here is the shape that pattern takes in practice. Imagine a 2,000-tweet pull in the first 48 hours after a product launch. By raw count the split reads 41 percent positive, 22 percent negative, the rest neutral, and a glance says the launch landed fine. Then you weight by likes and the positive share slides to 29 percent, because three sharply-worded negative threads pulled tens of thousands of likes between them while the praise was spread thin across small accounts. The raw number and the weighted number disagree, and the weighted one is the version your communications team should see. Treat these figures as an illustration of the mechanism rather than a measured result; your own numbers will depend entirely on the topic and window.

---

Step 7: Go async for large pulls

Cost is flat at $0.0008 per page no matter how you fetch, but wall-clock time is not. For tens of thousands of tweets, concurrency turns minutes of waiting into seconds. Cursor pagination is inherently sequential because each cursor depends on the previous response, so the win comes from overlapping the network round-trips and trimming the sleep. This is the part people most often get wrong, so it is worth stating plainly: within a single query you cannot parallelize the pages, because page two's cursor only exists after page one returns. The asyncio and aiohttp stack still helps, partly by shaving the per-request overhead and partly because the real parallelism is across queries, not within one. If you are monitoring five brands, fire five independent cursor walks concurrently and you get a near-fivefold speedup with no change to the per-query logic.

import asyncio
import aiohttp


async def fetch_page(session: aiohttp.ClientSession, args: dict) -> dict:
    endpoint = f"{ROOT}/twitter/tweet/advanced_search"
    async with session.get(endpoint, headers=AUTH, params=args) as resp:
        resp.raise_for_status()
        return await resp.json()


async def harvest_async(query: str, target: int = 10_000) -> list[dict]:
    """
    Pull about `target` tweets. ~20 land per page, so target // PER_PAGE pages.
    10,000 tweets == ~500 pages == $0.40 in read calls.
    """
    pages_needed = target // PER_PAGE
    collected: list[dict] = []
    args = {"query": query, "product": "Latest"}

    async with aiohttp.ClientSession() as session:
        for _ in range(pages_needed):
            payload = await fetch_page(session, dict(args))
            batch = payload.get("tweets", [])
            collected.extend(batch)
            cursor = payload.get("next_cursor")
            if not cursor:
                break
            args["cursor"] = cursor
            await asyncio.sleep(0.02)

    return collected[:target]


big_pull = asyncio.run(
    harvest_async('("product launch" OR "new feature") lang:en -is:retweet', target=10_000)
)
print(f"Pulled {len(big_pull)} tweets | spend ${spend(len(big_pull)):.4f}")

Five hundred sequential pages with a 20 ms gap finishes in a handful of seconds of wall time. The bill is the same estimated $0.40 it would be at any speed. Because billing is per call rather than per fixed time window, there is no quota reset to plan around and no 15-minute penalty box, which is the model the official platform uses. If you are coming from that world and your code has retry-and-backoff logic built for token-bucket rate limits, you can mostly delete it here; the only ceiling that exists is your account balance. The Twitter API rate limit guide walks through the difference in detail and is worth reading if your current pipeline is full of sleeps you added to dodge a 429 that no longer applies. A sane default is still to cap concurrency at 10 to 20 in-flight requests, not because a quota forces it but because hammering any endpoint with hundreds of simultaneous connections is rude and tends to surface flaky network errors that are annoying to debug.

---

What 10,000 tweets actually costs

The honest accounting, built on the canonical $0.0008-per-page, roughly-20-tweets-per-page rate:

Line item	Pages	Cost
10,000-tweet search (~20/page)	500	$0.40
Deduplication and retry headroom (~5%)	25	$0.02
Run total	525	$0.42

So a single 10,000-tweet sentiment run is about $0.40, which is the figure in the title, and the $0.50 of free signup credit covers it outright. Per volume that is $0.04 per 1,000 tweets. Scoring adds nothing on top: TextBlob and VADER are free CPU work, and the transformer is a free model whose only cost is the compute you already own.

For a reference point against the official X API, its lower paid tiers meter reads in the low thousands of posts per month for a three-figure monthly fee, so a single 500-page run can eat a noticeable slice of a monthly quota. Those numbers move, so treat that comparison as a rough model rather than a quote, and check the current X API product page before you plan around it. The full endpoint and pricing reference lists the read rate, the $0.0008 simple-write rate, the $0.0016 rate for tweet creation and DM send, and every one of the 48 endpoints.

The pricing gap is the entire economic case for a sentiment monitor at this scale, so it is worth seeing the comparison laid out properly rather than taking one paragraph's word for it. The 2026 X API pricing change explainer tracks how the official tiers shifted and why per-call pricing changes the math for high-volume read workloads, the Twitter API v2 versus twitterapis comparison puts the two surfaces side by side feature for feature, and the eight-provider cost ranking shows where each option lands on real per-1,000-tweet cost. If you are migrating an existing pipeline rather than starting fresh, the migration guide from twitterapi.io and the writeup on RapidAPI Twitter alternatives cover the practical swap. One thing the table above understates: the free-credit cushion means your first real run, including the inevitable false starts where you fix a query and re-pull, costs nothing, which removes the usual fear of "burning budget while debugging" that makes people under-test a data pipeline.

---

Picking a model on purpose

	TextBlob	VADER	twitter-roberta-base-sentiment
Install size	~1 MB	~1 MB	~500 MB
Speed on CPU	Under 1 ms	Under 1 ms	5 to 50 ms
F1 on SemEval-2017 tweets (estimated)	~58%	~62%	~72%
Emoji and slang	Weak	Strong	Strong
Sarcasm and irony	Poor	Poor	Better
GPU helps	No	No	Yes
Use it for	A quick sanity check	Real-time dashboards, huge volumes	Accuracy-critical reporting

A common production setup runs VADER across the entire pull for instant coverage, then reserves the transformer for the slice that actually needs precision: the high-engagement tweets and the ones where VADER's confidence sits near the neutral boundary. Routing only the ambiguous and amplified tweets through the transformer cuts its workload by roughly half with no measurable hit to the daily aggregate.

This split is not theoretical. One widely-shared community project analyzed nearly a thousand threads with exactly this pairing, Cardiff NLP's tweet-tuned RoBERTa for the long comments and VADER for the short titles, then published the per-product breakdown and the model's accuracy caveats:

Best headphones according to Reddit: I analyzed 998 posts (90K comments) using NLP sentiment analysis from r/headphones

---

Hardening the pipeline for production

The tutorial script is a foundation. A monitor you can leave running needs four more pieces. The gap between a notebook that produced one nice chart and a service that runs unattended for months is mostly about handling the cases that do not show up in a clean first pull, so it is worth treating each of these as a requirement rather than a nice-to-have. The twitterapis best practices guide collects the operational habits that keep a long-running poller healthy, and the points below are the sentiment-specific ones.

Builders have been wiring this exact kind of monitor together for years. One popular open-sourced project streams both Twitter and Reddit through Python, scores each mention, and surfaces tickers whose mention volume and sentiment are accelerating at the same time, which is the same two-signal read this post charts, just pointed at stocks instead of brands:

I built a program that tracks mentions and sentiment of stocks across Reddit and Twitter to find rising stocks from r/stocks

Broaden the query for recall. A bare "AcmeCorp" query under-counts the angriest mentions, because people misspell a brand when they are upset. Widen it: ("AcmeCorp" OR "Acme Corp" OR "@acme") lang:en -is:retweet. Compare total volume between the narrow and broad versions on the same window to confirm you are catching real signal and not noise.

Deduplicate across runs. Concurrent and overlapping fetches will return the same tweet twice, which double-counts it in engagement-weighted math. Keep a set of seen tweet IDs and drop repeats before scoring. A local SQLite store is enough to start:

import sqlite3

_db = sqlite3.connect("seen.db")
_db.execute("CREATE TABLE IF NOT EXISTS seen (id TEXT PRIMARY KEY)")


def fresh_only(tweets: list[dict]) -> list[dict]:
    out = []
    for tw in tweets:
        tid = str(tw.get("id"))
        try:
            _db.execute("INSERT INTO seen (id) VALUES (?)", (tid,))
            out.append(tw)
        except sqlite3.IntegrityError:
            pass   # already counted in a previous run
    _db.commit()
    return out

Swap SQLite for Redis once you are running multiple workers, since a shared set avoids the race where two workers both think a tweet is new.

Sample instead of scoring everything. If a brand throws off 50,000 mentions a day, you do not need to run the transformer on all of them. A random sample of 1,000 to 2,000 tweets per day holds the margin of error under about 3 percent at 95 percent confidence. Run VADER on the full firehose for anomaly detection and the transformer on the daily sample for the authoritative score.

Strip bots before they skew the score. Spam and reply-bot accounts pile onto trending tags and drag the aggregate toward whatever copy they are pushing. Two cheap filters remove most of them. At the query level, min_faves:1 drops the zero-engagement floor where most spam lives. At the analysis level, discard rows whose author shows the classic bot signature of zero followers paired with a high following count. If you need a cleaner sample still, add is:verified or a blue-check filter to the query and accept a smaller but higher-signal set. Bot contamination is the single most underrated source of bad sentiment numbers, because it does not look like an error, it looks like a real shift in mood, and it tends to spike exactly when a topic is hot and you most want to trust the chart. The heuristics here catch the crude botnets; for the subtler ones, the Twitter bot detection guide goes deeper into account-age, posting-cadence, and duplicate-text signals that the follower ratio alone misses. The practical rule is to run your bot filter, then eyeball the top ten most-liked tweets driving your aggregate; if any of them are obviously coordinated, your filter needs another pass before the number means anything.

def looks_human(tweet: dict) -> bool:
    author = tweet.get("author", {})
    followers = author.get("followers_count", 0)
    following = author.get("following_count", 0)
    if followers == 0 and following > 1000:
        return False            # zero-follower, mass-follow: typical bot
    return True


cleaned_sample = [tw for tw in big_pull if looks_human(tw)]

Schedule and alert. A cron or APScheduler job on a 15-minute cadence, a small table storing tweet ID, UTC timestamp, text, label, and score, and an alert that fires when the 3-day rolling mean drops below a threshold (say -0.3) gives you a brand monitor for an estimated $2 to $5 a month at typical volumes. Tune the threshold against past incidents so it would have fired within half an hour of a real crisis.

A minimal schema that supports every dashboard view: tweet ID (primary key), created_at in UTC, text truncated to 280 characters, the VADER compound score, the final label, and the query string that surfaced the tweet. Index on created_at for time-series queries and on label for distribution counts.

---

Things that bite people

The transformer returns mostly neutral. Usually the text was almost all links and handles, so cleaning left an empty string. The tidy(t) or t fallback in roberta_batch guards against that; confirm your cleaner is not eating the whole tweet.

You get a 429. Billing is per call with no fixed window, so a 429 here means the account balance hit zero, not that you tripped a rate limit. Top up from the dashboard and retry; the response body's message field spells out the reason.

VADER and the transformer disagree by a mile. Expected on sarcasm-heavy topics like politics, crypto, and tech criticism. Trust the transformer labels there and use VADER only as a fast first pass.

Pagination stops early. The cursor goes empty when no more results match inside the requested window. Widen the date range or relax an operator (drop min_faves, broaden the OR group) to recover depth.

Time-zone drift in the chart. Tweets are timestamped in UTC. Convert to local time before binning by hour or your "morning versus evening" story will be off by your offset.

---

Is analyzing public tweets allowed?

For public tweets, reading and scoring them is standard practice across academia and industry. The sentiment computation itself, turning public text into a number, is not the part that raises questions. What you do afterward is: storing tweet text long-term and republishing it are governed by the X Terms of Service and by data-protection law such as GDPR and CCPA, depending on where your users are. The 2022 hiQ Labs v. LinkedIn decision in the Ninth Circuit held that accessing publicly available web data does not by itself violate the Computer Fraud and Abuse Act, which is the case people usually cite here. None of this is legal advice; if you plan to retain raw tweet text or surface it in a product, talk to counsel and read the current developer terms before you ship. A safe default for most monitoring work is to store the score and a truncated snippet rather than full archives of tweet text.

Where to take it next

The advanced search reference documents every operator for filtering by language, region, engagement floor, verified status, and media type, which is how you raise sample quality before any model runs. The use cases page lays out polling cadences and deduplication patterns teams use in production. The full endpoint docs cover the write side, the simple toggles at $0.0008 per call and tweet creation at $0.0016, in case you want to close the loop and post an alert or auto-reply when sentiment crosses a line.

To turn this into a standing monitor, add the three pieces from the hardening section: a seen-ID store for deduplication, a scheduler for the poll, and a rolling-mean alert. With those wired in, the code above is a real brand-sentiment dashboard that runs for a few dollars a month, and you can extend it to topic clustering or per-region breakdowns whenever the basic trend line stops being enough.

If you would rather watch the whole pipeline assembled before you build your own, a full beginner walkthrough takes collection, cleaning, scoring, and charting end to end in one sitting, which is a useful way to sanity-check your mental model against someone else's working code.

https://www.youtube.com/watch?v=9ZUEhIUQ_FY