#FakeNews and a “Post-Fact” news cycle destroyed our sense of trust in news sources – especially in the digital space. The writers at FreePressFail courageously battle on a day-to-day basis to help the average American decipher all the lies and deceptive narratives that fly around on digital news. Today we thankfully recognize more people joining the fight.
www.fakenewschallenge.org has taken up the torch from the writers at FreePressFail and has posted “rules” for calling out #fakenews:
Post-facto Fake News Challenge
DescriptionPost-facto fake news refers to news items or claims that are already known to be false, either by work from organizations like Snopes and Factcheck or by the general public on social media.
For this challenge, we will only consider claims that are outright false or outright true. For example, “eating fruit prevents cancer” — a truth assertion here is dodgy at best, but for headlines like “Hillary has a body double” we can be confident about truth assertion.
We will select headlines for the competition where we can be confident in asserting its veracity.
A claim/headline string with accompanied URL, if any.
- “Pope endorsed Trump”
- “Climate change is a Martian takeover operation”
OutputFour required values
- 1. Boolean fakeness indicator
- 2. Confidence score
- 3. Provenance URL to support
- 4. A confidence threshold for accepting/rejecting
We will provide the training data
Teams will use part of the training data as a development set
Test data is withheld
In addition to training data, you are free to use all of the internet. The solution has to work in real-time.
Evaluation CriteriaSubmissions will be evaluated based on a 0-1 loss using the submission-provided threshold. All winning entries must provide a provenance URL for their solutions.
AwardsThere is a total of $2000 to be awarded for the top-5 teams.
Each team will get cash awards proportional to their accuracy.
Submissions achieving more than 10% false positive or false negative rates are not considered.
Contesting teams must be willing to open source their systems.
TimelineTeams register by: Jan 1st, 2017
1st batch of training data released: Jan 15th, 2017
2nd batch of training data released: Feb 15th, 2017
Submissions due: June 1st, 2017