lately

I built a detector for fake podcast voices

Brian Stever

Deepfake Audio Detection · 2026 · PyTorch, torchaudio, ASVspoof, spectrogram CNNs

The short version. This started as coursework and wandered straight into familiar territory: podcast production, where a voice is evidence as well as audio. I built a synthetic speech detector around log-Mel spectrograms, a compact CNN with Squeeze-and-Excitation attention, and ASVspoof evaluation data. The final model is fast and genuinely useful as a research prototype. It isn't ready to guard the studio door by itself, and that's a much better finding than pretending otherwise.

1Why podcast voices

I've spent eleven years making podcasts, and podcasting does a strange thing to voices. A host's voice becomes a brand, a relationship, a production cue and, occasionally, a shaky little authentication system wearing headphones. If a clip sounds like the right person, everyone is tempted to relax.

Deepfake audio makes that instinct risky. The attacker I'm imagining doesn't need a movie-villain setup. Public episodes, guest spots, livestreams and social clips can all become source material, and the fake can arrive looking like one more file in the publishing flow. This detector isn't the whole security plan. It's the extra person in the room asking, politely, whether that voice is really who it sounds like.

2The pipeline

Every clip is converted to mono 16 kHz audio, padded or trimmed to four seconds, and turned into a 128-bin log-Mel spectrogram. The main model treats that spectrogram as a single-channel image and runs it through a four-block CNN with Squeeze-and-Excitation channel attention.

Training used the ASVspoof 2019 Logical Access train and dev splits. Evaluation then moved to the full ASVspoof 2021 LA set, and that jump matters. On the friendly split, a model can look like it's been lifting weights. The 2021 benchmark asks it to carry furniture up the stairs.

Sample audio spectrogram used by the deepfake audio detector
Figure 1. A sample spectrogram from the preprocessing pipeline. The detector isn't listening the way a person would. It's looking for suspicious time-frequency artifacts.

3How it did

The main CNN + SE model reached 18.28% EER and 0.8979 AUC-ROC on the full ASVspoof 2021 LA evaluation set. Its F1 at a fixed threshold was strong, but EER is the more security-relevant metric, and it keeps the story honest. Working detector, yes. Deployable countermeasure, not yet.

A podcast-style stress test added noise, room response and MP3-like compression. The model got worse, because audio production is where clean signals go to develop a personality, but it didn't fall apart. EER moved from 18.28% to 18.48%, and AUC dropped from 0.8979 to 0.8852.

Table 1. Selected evaluation results.

ExperimentEERAUCF1
CNN + SE, clean 2021 LA18.28%0.89790.9479
CNN + SE, podcast stress18.48%0.88520.9430
ResNet-18 baseline17.21%0.89570.9145
LFCC-GMM baseline30.89%0.74000.8128
Bar chart comparing EER results for deepfake audio detection models
Figure 2. Equal Error Rate for the main CNN + SE model, the podcast stress test, ResNet-18 and the LFCC-GMM baseline. Lower is better.

4Against the baselines

ResNet-18 had the best EER, at 17.21%, so the smaller model doesn't get a parade. The more interesting result is a practical one: CNN + SE stayed competitive with far fewer parameters, and it ran faster in local timing on a MacBook.

The LFCC-GMM baseline was much weaker, at 30.89% EER. That gave the project a useful floor. A traditional cepstral-feature approach wasn't enough for this version of the task, and the learned spectrogram models handled the benchmark much better.

ROC curve for the CNN plus Squeeze-and-Excitation deepfake audio model
Figure 3. ROC curve for the CNN + SE model on ASVspoof 2021 LA. Good enough to be interesting, and not good enough to pretend the problem is solved.

5What it showed

As engineering, it worked. Data loading, preprocessing, augmentation, training, checkpointing, baseline comparison, evaluation, plotting and metric export all ran end to end. As research, it failed in a useful way: the 2021 benchmark exposed a generalization gap that the 2019 development split had politely kept hidden.

That's probably the right lesson. A detector can look heroic in the lab and still get weird when codecs, channels and synthesis systems shift. The next version would need threshold calibration, stronger anti-spoofing architectures, and tests on real podcast export chains built from the tools production teams actually use.