Overview

This project covers the complete journey of building a data product—from an initial personal loss on a disputed market, to reverse-engineering cross-chain protocols, designing a Bayesian scoring engine, and deploying the live platform on self-hosted infrastructure.

screenshot-1788608376612

Screenshot_5-9-2026_1880_polydispute.oliverq.io

If you came from my LinkedIn article, you can skip past Section 1 explaining why I built Polydispute and what problem it solves.

In building Polydispute, I had to wear several hats. This post is structured around each of those roles:

  1. Business Problem: Understanding Polymarket dispute mechanisms, voter incentives, and where retail capital gets trapped.
  2. Data Modelling: Sourcing data across disparate APIs, establishing relational entity models, and designing state-driven ingestion pipelines.
  3. Statistics: Calibrating Discord commenter credibility through Bayesian averages and point-in-time scoring.
  4. Infra and Ops: Making pragmatic infrastructure trade-offs (Hetzner, Coolify, MotherDuck, Doppler, Prefect, Sentry) to run realtime analytics on a $10/month budget.
  5. Full-Stack Development: Building a responsive, read-heavy React dashboard and FastAPI backend with caching.
  6. Trading: A reality check on paper accuracy vs. execution risks, illiquidity, and whales.
  7. Product Management: Unimplemented features, including automated arbitrage alerts and LLM-powered evidence synthesis.

Glossary

Before diving into the architecture, here is a quick reference for terms I use throughout this article:

  • Market: A Polymarket market that resolves to either YES (P2) or NO (P1). In rare cases, a market can resolve to 50-50 UNKNOWN (P3), but this is an exception.
  • Dispute: A process in Polymarket in which a market resolution is proposed and someone raises a dispute against said proposal. Disputes last 24-48 hours and are resolved by UMA DVM to one of 4 outcomes (see below)
  • P1, P2, P3, P4: The discrete resolution outcomes in UMA. The price of the YES token settles to a $ amount depending on the resolution outcome:
    • P1 = NO ($0.00)
    • P2 = YES ($1.00)
    • P3 = 50-50 Indeterminate ($0.50)
    • P4 = Too Early (rejection of proposal; contract returns to open trading)
  • Event: A Polymarket ‘container’ for multiple related markets (e.g., “Who will win the Super Bowl?” contains unique markets “Giants will win the Super Bowl?”, “49ers will win the Super Bowl?”, and so on). While events provide useful metadata; the core entity is always the individual market.
  • Voter / Discord User: A commenter in the UMA Discord #disputes channel who states an outcome preference. In this project, I use voter and Discord user interchangeably, but strictly speaking, true voters are the on-chain UMA tokenholders voting on the Ethereum Mainnet. Since individual wallet-to-Discord identity attribution is virtually impossible, we treat Discord commenters as a proxy for voter sentiment.
  • Market Price: The current orderbook price of the YES token. If the market resolves YES, each YES token pays out $1.00; if it resolves NO , it expires at $0.00. We track the YES price exclusively because YES and NO prices are inversely related.
  • Accuracy Score: The Bayesian accuracy rating of a forecaster (voter). It is a composite of a user’s raw win rate with a prior win rate (50%) to penalize inexperienced voters.
  • DVM (Data Verification Mechanism): UMA’s decentralized dispute oracle on Ethereum Mainnet where UMA tokenholders vote to settle contested assertions.
  • Ancillary Data: A hex-encoded byte string containing the Polymarket resolution rules, question text, and criteria submitted when creating or disputing a market. This is used to link transactions across different blockchains.

System Architecture

image-20260906154010370

At a high level, this is a data application that operates across three layers:

  1. Ingestion Layer: A state-driven Prefect pipeline polls Discord (#disputes), Polymarket Gamma (metadata), and Polymarket CLOB (orderbook tick history).
  2. Analytical Warehouse:
    1. Pipeline writes data: Ingested data is cleaned and stored in MotherDuck (serverless cloud DuckDB).
    2. App reads data: Point-in-time user believability scores and market aggregations are calculated dynamically via SQL views and functions.
  3. Serving & Application Layer: A single Docker container hosts FastAPI to serve JSON API contracts and static React assets.

Business Problem

Unexpected dispute resolutions on Polymarket have cost bettors millions of dollars - be it due to a technicality in the market’s rules and wordings or outright fraudulent resolution proposals.

Disputes are common but there isn’t much clarity on how the process works or who actually determines the resolution of a market. While WSJ recently covered this lesser-known part of prediction markets, I had already been burned long before by disputes resolving against the general market consensus.

Disputing a Market

In true Web3 fashion, the dispute resolution process is decentralized —which also means it is messy.

The Polymarket dispute and resolution workflow involves a multi-chain transaction across Polygon and Ethereum Mainnet. Documentation across the various contracts and protocol bridges is fragmented, making it difficult for everyday retail traders to understand where their money actually goes when an outcome is contested.

When a Polymarket event proposal is disputed, it escalates to UMA’s Data Verification Mechanism (DVM). Crucially, this voting process does not happen on Polygon (where trading occurs); it settles on Ethereum Mainnet, governed by UMA tokenholders.

I have added a diagram to outline the basic interactions. However, knowledge of the resolution process isn’t relevant to the project.

image-20260906154650852

Discord Dispute Discussions

Disputes operate as a Keynesian Beauty Contest: voters are not economically incentivized to vote for the “objective truth” in a philosophical sense. Instead, they are rewarded for voting with the majority consensus, and penalized via losing tokens if they vote against it. Furthermore, token holders can delegate voting power to committees. For instance, UMA Rocks is a 3-person committee voting on behalf of 39 holders, controlling ~9% of the voting pool.

My initial objective was to map individual Ethereum voting wallets back to specific Discord usernames to directly correlate chat sentiment with on-chain voting power. When that proved impossible, I pivoted to treating Discord as a standalone predictive market tournament.

Problem Solving

This pushed me to build Polydispute and in the process answer 2 questions:

Could you track which resolution voters were gravitating towards in Discord channels?

Could you trade the difference in this vote consensus vs. the current market price before the final vote is revealed?

Data Modelling

Designing the data architecture was handled as 3 problems:

  1. Identifying and mapping all available data sources across Polymarket, Discord, and UMA.
  2. Structuring models that link Discord chat sentiment to Polymarket market contracts.
  3. Designing state-driven pipeline ingestion to ensure idempotency while respecting API rate limits.

Data Sourcing

This was not a clean, linear process. Every time I ran into an API limitation or incomplete dataset, I had to iterate and find proxies for that missing data.

For instance, the Polymarket Gamma API occasionally returns resolved markets with closed=False. Catching these edge cases was critical for downstream analytics to prevent closed markets from perpetually consuming pipeline polling cycles.

I ultimately settled on four primary API sources:

API Purpose Routes & Response Considerations
Discord REST Dispute threads & community messages GET /channels/{id}/threads/active
GET /channels/{id}/messages		 Market IDs are parsed from the thread’s initial message.
Regex extracts stance tokens (P1–P4) from unstructured chat.
Only the latest valid vote per user is retained.
Authentication requires a persistent bot/user token.
Polymarket Gamma Market metadata, rules, token IDs GET /markets?id={market_id}
Returns JSON market object		 Provides question text, category, outcome token IDs (clobTokenIds), and resolution criteria.
Requires secondary checks against settlement prices to catch stale closed=False flags.
Polymarket CLOB 1-minute midpoint price histories GET /prices-history?market={token_id}&interval=1m
Returns timestamps & midpoints		 Granular data (1,440 points per market/day).
The pipeline specifies a time range dispute_start and resolution_time to eliminate redundant data.
UMA Rocks Vote Committee consensus & pre-commitments GET /api/getPoolAnswers
Returns JSON array of answers		 UMA Rocks voting pool answers (P1–P4) appear here before they commit it to the blockchain, serving as an institutional signal.

Data Model

All ingested entities link back to a unique Polymarket market_id.

Below is a truncated ER diagram that shows the main columns in each table. Additional columns which are not related to other tables are not shown. You can see the full schema in Github.

image-20260906154945055

  • The core entity is Market (pm_markets).

  • A Thread (dc_threads) is created when a market enters dispute.

  • A Thread contains multiple Messages (dc_messages) submitted by AuthorUsernames (Discord users)

  • In parallel, each Market maintains a PriceHistory (pm_price_history) table recording 1-minute YES midpoint ticks throughout the dispute window.

  • Tables are organized in raw > clean > vw progression for better segregation between data of different quality.

    Further improvement could be made to only use cleaned, analytics-tier data for the web app consumption, but this had minimal effect at the current scale and hence the app does use data from append-only raw tables like (price_history)

Side Note on Blockchain Data

I initially planned to query the Ethereum Mainnet directly (via RPC or Goldsky GraphQL subgraphs) to extract exact on-chain vote distributions. However, this introduced substantial operational overhead for marginal benefit.

Keccak256 AncillaryData (Polygon to Ethereum Bridge) When trying to link Polymarket’s Polygon smart contract (UmaCtfAdapter) to Ethereum Mainnet DVM votes, raw string joins on ancillaryData failed completely. On Polygon MOOv2, ancillaryData is stored as the full raw hex string of the market rules. But when bridged to Ethereum Mainnet DVM, UMA hashes the data into a compressed format:

ancillaryDataHash:<keccak256_of_original>,childBlockNumber:<block>,childOracle:<addr>,childChainId:137

I had to either compute keccak256(polygon_ancillaryData) locally to match the hash prefix, or bypass ancillaryData string matching entirely by querying UMA’s subgraph using (identifier, time) composite tuples.

I found that once a market settles, Polymarket’s Gamma and CLOB APIs provide reliable ground-truth resolutions directly. Bypassing direct Ethereum node queries simplified the pipeline while keeping data consistent with orderbook outcomes.

Treating Discord as a Prediction Tournament There is no way to link a Discord user to their crypto wallets, so I chose to use Discord as a proxy. We don’t care about a user’s wallet balance; we care about their track record on previous disputes. If a user consistently calls disputes correctly before on-chain reveals, their opinion carries statistical weight regardless of how many tokens they hold.

Ingestion Logic

A major architectural change was moving away from traditional rolling time-window ingestion toward entity-lifecycle ingestion.

In standard batch pipelines, jobs query fixed rolling intervals across all endpoints:

  • Fetch all Discord messages from t_0 to t_1.
  • Fetch all Polymarket markets from t_0 to t_1.
  • Fetch price histories from t_0 to t_1.

While a time-range approach is unavoidable for Discord message streams, applying it to Polymarket is inefficient:

  1. Polymarket lists thousands of active markets each day, and the API does not allow filtering by dispute status.
  2. Indiscriminately pulling 1-minute price ticks across all markets produces $24 \times 60 = 1,440$ rows per market per day. Trying to poll price history for markets that are closed will lead to millions of unchanged data.

The better solution was to isolate ingestion strictly to markets currently in dispute. This led to a pipeline sequence which will:

  1. Poll Discord Threads: Ingestion begins by querying active threads in the #disputes channel. This immediately narrows the operational catalog from tens of thousands of markets down to only ~ 5-10 new disputed markets a day.
  2. Catalog Upsert & State Transitions:
    • If a market is new, fetch its metadata from Polymarket API and insert it into pm_markets with status DISPUTED.
    • If it already exists, inspect whether its status has transitioned from DISPUTED to RESOLVED / CLOSED.
  3. Bounded Price History Pulls:
    • For markets actively in dispute:
      • if it is the first time seeing the market, pull the entire price history since the market’s start_date
      • for markets that are already being monitored, fetch price history across the active dispute window using a NOW() - TIME_INTERVAL range to avoid pulling too much data.
    • Once a market resolves, pull the final price history between dispute_start and resolution_timestamp and avoid future ingestion for that market.

Even though the pipeline can handle parallel, asynchronous HTTP calls at high volumes, this entity-state design makes it more efficient. By limiting the markets that need to be updated every 5 minutes, we minimize API calls, redundant database writes, and guarantee idempotency.

The entire pipeline executes in under 60 seconds, allowing us to run on a 5-minute schedule without job overlap.

image-20260906155232299

Statistics: User Calibration

Using Discord comments as a proxy for voting consensus introduces a signal-to-noise problem: how much weight should be assigned to an individual commenter?

Should a newcomer who is 5-for-5 (100% win rate) outrank an experienced forecaster who is 9-for-10 (90% win rate)?

This is similar to e-commerce product ranking or chess ratings: we need an algorithm that balances small-sample variance against historical performance by regressing scores toward a global prior. (This article on Bayesian Averages explains the concept).

In this system, a commenter’s credibility score dictates their voting weight in the aggregate consensus for an active dispute. To aggressively reward proven forecasters and suppress noise, we square the calibrated score weight = score^2 rather than applying a linear weight.

Bayesian Average

To balance sample size against raw performance, we use Empirical Bayes shrinkage:

Bayesian_Score = (correct_preds + N * system_accuracy) / (all_preds + N)

Where:

  • correct_preds = Total correct predictions made by the user
  • all_preds = Total gradeable disputes the user voted on
  • system_accuracy = System prior accuracy = 50% (assume a new user is no better than 50/50 guessing)
  • N = Trust number = 20 (number of disputes needed to pull a user away from the system accuracy)

This is a baseline configuration which can be changed to tune the grading of user accuracy.

If a new user arrives and consistently maintains a 95% win rate, their Bayesian accuracy climbs gradually toward their raw win rate:

image-20260906161231950

To reach a 90% Bayesian score, a user with a sustained 95% win rate must participate in 160 disputes:

Returning to our earlier comparison:

  • User 1 (5-0): 100% raw hit rate = 60.0% Bayesian score.
  • User 2 (9-1): 90% raw hit rate = 63.3% Bayesian score.

Even though User 1 has a ‘flawless’ 100% win rate, User 2 scores higher because 10 disputes provide twice as much evidence of skill than a 5-dispute lucky streak. Their scores are still very low at ~60%, showing that users need hundreds of disputes under their belt to be considered qualified in their forecasts.

(Failed) Trading Signals

In earlier iterations, I tried converting the 4-way consensus distribution (% share of P1, P2, P3, P4) into a single trading signal between 0.00 and 1.00 by computing a weighted Expected Value upon dispute resolution:

Expected_Value = (%_P1 * Payoff_P1) + 
									(%_P2 * Payoff_P2) + 
									(%_P3 * Payoff_P3) + 
									(%_P4 * Payoff_P4)

Where payoffs were mapped to resolution values:

  • P1 (NO) = $0.00
  • P2 (YES) = $1.00
  • P3 (50-50 Unknown) = $0.50
  • P4 (EARLY) = YES Price(NOW)

What is the payoff of a P4 (Too Early) resolution? UMA voters vote P4, they are rejecting the settlement proposal and returning the contract to open trading to the current market price at the moment the vote was cast:

Payoff_P4 = Current_Market_Price_YES

While this fixed the value calculation problem, I ultimately moved away from an automated EV signal because the small sample of disputes where the payoff could be reliably calculated is too small to be useful.

Point-in-Time Scores

A critical requirement when backtesting or analyzing historical markets is ensuring zero lookahead bias.

When a user posts a vote on Dispute B at timestamp t, their believability weight must be computed only from disputes that settled before t. You cannot use a user’s lifetime accuracy today to weight consensus for a Dispute A that happened six months ago.

Because believability evolves dynamically with each market, user scores cannot be pre-computed. They must be evaluated on the fly per dispute market.

Simply put, you have to evaluate a user’s credibility based on their performance in the past, at that past point in time, rather than based on their current performance record. This goes into bi-temporal data modelling which others are better at explaining.

[!NOTE]

This requirement heavily influenced my database choice: we needed a database capable of executing dynamic, vectorized joins across millions of rows without pre-aggregating views.

Individual forecaster metrics can be inspected on the Voter Calibration page:

image-20260906161023202

Infra and Ops

While enterprise tools aren’t strictly required for a personal side project, I intentionally designed Polydispute as an end-to-end production template. Having a reproducible blueprint allows me to reuse this formula across future data-intensive platforms.

Coming from an AWS background, I specifically wanted to explore lightweight, self-hosted alternatives:

Component AWS Equivalent Self-Hosted / Lightweight Stack
Compute EC2 Hetzner Cloud VPS
Deployment ECS Coolify
Container Registry ECR GitHub Packages (GHCR)
CI/CD CodePipeline GitHub Actions Workflows
Orchestration Step Functions / MWAA Prefect
Observability CloudWatch Sentry APM
Reverse Proxy ALB Traefik (Coolify managed)
Secrets Management AWS Secrets Manager Doppler CLI
Data Warehouse Redshift MotherDuck (Cloud DuckDB)
Storage S3 Cloudflare R2 (dev testing)

The entire application and pipeline runs on a $10/month Hetzner VPS, backed by a free MotherDuck instance, Doppler for secrets injection, Sentry for APM, and GitHub Actions for CI/CD.

Deployment is managed via Coolify on the VPS, which automates Docker Compose deployments and networking:

image-20260906161817451

Pipeline orchestration is handled by a self-hosted Prefect server running on the same VPS. The Prefect worker also serves as the compute environment for the pipeline jobs that it runs.

Why In-Process Prefect Worker? Standard Prefect deployments frequently use DockerWorker to spin up an ephemeral container for every flow run. However, spinning up a fresh container adds 10-15 seconds of startup latency and burns memory. On a 5-minute run schedule, this risks exhausting the resources I have on my $10 VPS.

Because the custom pipeline image already bundles all required dependencies (duckdb, pydantic, httpx), running the worker in-process executes jobs in under 200ms with no container overhead.

Database: Why MotherDuck?

I selected MotherDuck (serverless cloud DuckDB) over alternatives like Neon (serverless Postgres) or local SQLite.

  • Why not local SQLite? The volume of 1-minute orderbook ticks was substantial (~19 million rows), and coupling local database files with pipeline worker containers prevented clean decoupled deployments.
  • Why not Postgres? Managing connection pools, index maintenance, and vacuuming on tick-level analytical queries on a small Postgres instance produced 800ms+ response times.
  • Why serverless DuckDB? DuckDB’s vectorized columnar execution handles cross-table analytical aggregations in sub-50ms. MotherDuck provides this in the cloud with zero database administration and a 10GB free tier.

With ~5 years of historical Polymarket dispute data, the database currently sits at ~7GB (out of 10GB free), with the vast majority consumed by 1-minute price ticks.

Downsampling older settled markets to 5-minute bars would easily drop this below 2GB.

image-20260906162138345

Caching and Secrets Management

Because MotherDuck’s free tier has monthly compute allowances, frequent pipeline runs every 5 minutes require cautious query budgeting.

To prevent frontend refreshes from repeatedly hitting the database, the FastAPI backend implements in-memory TTL caching to match the data refresh rate of 5 mins (300s):

Secrets are managed entirely via Doppler CLI. At container startup, Doppler injects credentials directly into process memory (doppler run -- uvicorn ...). This avoids plaintext .env files or hardcoding ENV_VAR in multiple locations.

image-20260906162345729

Application monitoring is wired via Sentry’s Python SDK, capturing transaction latencies, database execution times, and client IP resolution through Traefik proxy headers.

image-20260906162445502

Full-Stack Development

The web application is intentionally designed as a read-heavy, low-overhead interface. The heavy analytical lifting occurs in DuckDB, so the FastAPI backend only needs to expose aggregation endpoints per market.

The frontend is built with React and bundled into the single Docker container.

A full demo of the platform in action is captured below:

Navigating the screener and main market info

demo1

Tracing price vs. vote consensus over dispute window

demo2

Exploring user comments by their vote stance and accuracy

demo3

The primary interface feature is the Consensus vs. Price Trajectory Chart, which overlays the live YES market price against the Bayesian-weighted vote consensus across all four outcomes (YES, NO, UNKNOWN, EARLY).

A key UX feature is interactive voter exploration: clicking on any point in the voter trajectory highlights the specific Discord discussion comments submitted at that timestamp, letting users read the arguments that caused the consensus line to move.

Trading

Across the entire dataset of ~2,000 historical disputes, qualified Discord users arrived at a consensus that matched the actual on-chain resolution 97% of the time.

However, paper accuracy does not automatically translate into an executable trading edge. I would not trade this signal mechanically for several practical reasons:

  1. Discord is an imperfect proxy: Chat commenters ≠ tokenholders. Whales and voting committees can remain completely silent in Discord and flip the outcome with their large token holdings.
  2. Dispute windows are fast: Uncontentious disputes resolve quickly without extensive public debate. By the time a high-confidence consensus forms in chat, the mispricing has usually closed.
  3. Orderbook illiquidity: Most disputed markets frequently suffer from thin orderbook depth and wide bid-ask spreads outweigh the profit to be made from trading the current price.
  4. Binary outcome volatility: A 0–1 Expected Value signal sounds neat in theory, but contracts resolve to binary $0.00 or $1.00 so small estimation errors can lead to extremely different outcomes.

Product Management

Although this data isn’t tradeable in the current iteration, there are still new features that would make the platform a useful tool.

If I were to continue developing Polydispute into an actionable trading product, these are the features I would explore next:

1. Automated Arbitrage Alerts

Setting up Webhook / Telegram alerts that trigger when:

  • A new dispute thread opens on Discord.
  • The spread between voter consensus and market price exceeds a defined threshold.

2. LLM Evidence Synthesis

In contentious disputes, Discord commenters frequently share authoritative evidence—linking to government press releases, video timestamps, or legal rule definitions. These are often the exact same sources UMA voting committees review.

Using an LLM agent to extract, verify, and summarize these external links in real time would give traders an instant breakdown of the underlying factual debate without having to scroll through hundreds of chat messages.

image-20260906163032571

Closing

Despite not walking away with an automated money-printing trading bot, building Polydispute provided immense engineering value.

  • Designing E2E data systems: This project reinforced that the most demanding aspect of alternative data systems is establishing sensible data models and calibration logic that protect downstream analytics from noisy, deceptive crowds.

  • Learning new infra platforms: Stepping outside the standard AWS ecosystem to self-host the entire architecture for $10/month gave me a portable, reproducible foundation that I can carry into future data products.

  • Research into prediction markets: Learning about the various loopholes in dispute resolution made me realise that there could be many more gaps in the market that people aren’t paying attention to. These are exciting areas to continue exploring.