Misinformation and disinformation in the media is a big deal these days. We're proud to have contributed to addressing this by building Verdad.app for our client Wayne State University.

It listens to thousands of hours of radio broadcasts in Spanish in the U.S. and listens using AI driven nuance for misinformation in that language. Then it extracts the relevant content, translates it into English, describes why it was selected, and provides access to the original content.

This is a tool built for journalists and newsrooms. But it is now available for everyone. 

Check out https://verdad.app.

VERDAD is an open-source platform designed to detect and analyze potential disinformation in radio broadcasts (across any language, but with a primary focus on Spanish and Arabic). Our multi-stage AI pipeline powered primarily by Google's multimodal Gemini LLM APIs and OpenAI Whisper speech-to-text (ASR) API. The audio recording pipeline is orchestrated by Prefect running on Fly.io. This system includes continuous audio recording, preliminary detection by a multimodal model, detailed transcription with timestamps, audio clip generation, and nuanced content analysis to generate structured output that is finally stored in a Postgres database and displayed in an interactive front-end where support journalists and researchers can review suspected snippets of mis/disinformation upvote the existing labels or add their own custom labels and discuss with each other in comments. Their feedback will also be used to improve the heuristics for flagging suspected mis/disinfo more accurately going forward.

The project's big goals are:

1. to provide visibility and data that can be used by trustworthy journalists to help disseminate timely factchecking and accurate counter narratives in response to specific mis/disinformation patterns;
2. to support investigations (big and small) into trends of mis/disinformation campaigns across regions and over time.

Project Overview

The VERDAD project addresses the critical challenge of monitoring and analyzing potential disinformation on radio stations who are targetted at immigrant communities. This medium represents a significant vector for the spread of mis/disinformation but it has received less systematic scrutiny than social media platforms.

Key Features

• Continuous recording and monitoring of Spanish-language radio stations
• Multi-stage AI analysis pipeline for disinformation detection
• Language-aware content analysis with cultural context understanding
• Collaborative platform for analysts to review and validate findings
• Structured data output for further research and analysis

Technical Architecture

VERDAD employs a multi-stage pipeline architecture designed for scalability and accuracy:

Stage 1: Initial Disinformation Detection

Input:

• Full audio file (5-15 minutes)
• Metadata (station info, timestamps, etc.)

Process:

• Uses Gemini 1.5 Flash LLM for rapid initial screening
• Applies simplified heuristics for high-recall detection
• Generates timestamped transcriptions using OpenAI Whisper

Output:

• Flagged snippet timestamps
• Basic categorization
• Initial confidence scores

Stage 2: Audio Clipping

Input:

• Original audio files
• Timestamps from Stage 1

Process:

• Extracts audio segments corresponding to flagged content
• Includes configurable context windows (before/after)
• Processes metadata for segment identification

Output:

• Individual audio clips for each flagged segment
• Structured metadata for each clip
• Storage paths and reference data

Stage 3: In-Depth Analysis

Input:

• Extracted audio clips
• Stage 1 metadata and categorization
• Cultural context data

Process:

• Utilizes Gemini 1.5 Pro LLM for detailed analysis
• Performs multi-dimensional content evaluation
• Generates comprehensive annotations

Output: Structured JSON including:

• Detailed transcription and translation
• Disinformation category analysis
• Confidence scores
• Emotional tone analysis
• Political leaning assessment
• Cultural context notes

Data Schema

Audio Files Table

audio_files {  id: uuid  radio_station_name: string  radio_station_code: string  location_state: string  recorded_at: timestamp  recording_day_of_week: string  file_path: string  file_size: integer  status: enum['New', 'Processing', 'Processed', 'Error']  error_message: string? }

Stage 1 LLM Responses Table

stage_1_llm_responses {  id: uuid  audio_file: foreign_key(audio_files)  initial_transcription: jsonb  initial_detection_result: jsonb  timestamped_transcription: jsonb  detection_result: jsonb  status: enum['New', 'Processing', 'Processed', 'Error']  error_message: string? }

Snippets Table

snippets {  id: uuid  audio_file: foreign_key(audio_files)  stage_1_llm_response: foreign_key(stage_1_llm_responses)  file_path: string  file_size: integer  recorded_at: timestamp  duration: interval  start_time: interval  end_time: interval  transcription: jsonb  translation: jsonb  title: jsonb  summary: jsonb  explanation: jsonb  disinformation_categories: jsonb[]  keywords_detected: string[]  language: jsonb  confidence_scores: jsonb  emotional_tone: jsonb[]  context: jsonb  political_leaning: jsonb  status: enum['New', 'Processing', 'Processed', 'Error']  error_message: string? }

Key Components

Recording System

• Supports multiple radio station formats
• Configurable recording durations and quality settings
• Robust error handling and recovery
• Cloud storage integration (Cloudflare R2)

Analysis Pipeline

• Language-agnostic design with Spanish/Arabic priority
• Cultural context awareness
• Continuous learning from analyst feedback
• Structured output for research use