Community Archive logo
Agent-friendly documentation

Build with the archive

Download the daily data export, query public records through the API, or give an agent one canonical starting point.

The Supabase REST API is retired

The public Supabase REST API was retired on 19 November 2026: the tweet corpus no longer answers the anon key. Use the keyed Community Archive API at https://analytics.community-archive.org/v1 for filtered queries and the daily export for bulk data.

Point an agent here

https://www.community-archive.org/llms.txt

This plain-text index includes the data dump, API v1 routes and keys, query examples, schema notes, and deeper documentation links.

If you use the archive, please cite it

This applies to the export, the API and anything an agent writes from them. Use this citation, with the date you accessed the data:

Carvalho, F. (2024). Community Archive [Data set]. https://www.community-archive.org (accessed [date you accessed it])

Choose an access path

Choose the policy-safe surface that fits your query.

Bulk Parquet export

Download tweets and profiles for bulk analysis. Each daily export checks current membership and opt-outs before publication.

One raw archive

Signed-in owners can download their own processed archive JSON while current PostgreSQL policy permits it.

https://www.community-archive.org/api/archive/<username>

Bulk Parquet export

The daily package contains enriched tweets.parquet, separate profiles.parquet, and a manifest.json with row counts, schemas, and checksums. It includes eligible members and applies current consent to referenced authors too.

Use the latest release link below to find the most recent successful export. GitHub lists the download links; the files stay in consent-managed storage. Superseded packages are removed, so bookmark the latest release rather than an individual Parquet URL. Downloads may be temporarily withdrawn when consent changes.

What can I build or discover?

  • Find a remembered quote, reply, link, or old discussion.
  • Revisit your own themes, collaborators, and changing interests.
  • Follow emerging ideas with Trends, Bangers, and the Digest.
  • Trace people, projects, events, and intellectual lineages.
  • Build a personal canon, visualization, or research dataset.
  • Analyze public interaction networks and communities.
More examples and starting points

API quickstart

Sign in, create a key on the API keys page, and send it as Authorization: Bearer cak_... on every request to https://analytics.community-archive.org/v1. Keep the key on your server or in your script, never in browser code. The full contract is the OpenAPI document.

cURL

export CA_API_KEY='cak_...'   # create one at https://www.community-archive.org/settings/api-keys

# One account's tweets, newest first (page with next_cursor until it is null)
curl --get "https://analytics.community-archive.org/v1/accounts/exgenesis/tweets" \
  -H "Authorization: Bearer $CA_API_KEY" \
  --data-urlencode "order=desc" \
  --data-urlencode "limit=5"

# Full-text search
curl --get "https://analytics.community-archive.org/v1/tweets/search" \
  -H "Authorization: Bearer $CA_API_KEY" \
  --data-urlencode "q=community archive" \
  --data-urlencode "limit=5"

JavaScript

const API = 'https://analytics.community-archive.org/v1'
const headers = { Authorization: `Bearer ${process.env.CA_API_KEY}` }

// Every tweet from one account, following the cursor
let cursor = null
do {
  const url = new URL(`${API}/accounts/exgenesis/tweets`)
  url.searchParams.set('order', 'asc')
  url.searchParams.set('limit', '200')
  if (cursor) url.searchParams.set('cursor', cursor)

  const response = await fetch(url, { headers })
  if (response.status === 429 || response.status === 503) {
    const wait = Number(response.headers.get('Retry-After') ?? '5')
    await new Promise((resolve) => setTimeout(resolve, wait * 1000))
    continue
  }
  if (!response.ok) throw new Error(`${response.status} ${await response.text()}`)

  const page = await response.json()
  for (const tweet of page.data) console.log(tweet.created_at, tweet.full_text)
  cursor = page.next_cursor
} while (cursor)

Routes

  • GET /v1/tweets/{tweet_id} — one tweet; ?ids= for up to 100
  • GET /v1/tweets?since=&until= — every tweet in a window of up to 31 days
  • GET /v1/tweets/search?q= — full-text search
  • GET /v1/accounts/{account} — profile by ID or username; /tweets, /likes, /followers, /following beneath it
  • GET /v1/members — every current member
  • GET /v1/me — your key and its limits

Rules

  • Responses are {data, next_cursor, request_id}. Page until next_cursor is null, not until a page is short.
  • Limits: 60 requests a minute and 20,000 a day per key, two in flight per account. On 429 or 503, wait Retry-After seconds and retry.
  • A 422 means the request was too broad for its query budget; narrow it, for example with a date range.
  • Treat Twitter IDs as strings, not numbers.
  • Deleted and opted-out tweets are omitted, so pages can hold fewer items than limit.

Retired: the Supabase REST API

Until 19 November 2026 the corpus tables were readable through Supabase PostgREST with a shared public anon key. That access has ended: tweets, likes, follows, mentions and the views and functions over them now refuse the anon key. Move to API v1 above with the call-by-call migration guide, or download the daily export for bulk work.

Deeper docs

The repository contains schema guidance and complete examples.