Connect with us

Fake News

#FakeNewsChallenge Trending on Twitter – Read Rules Here

Published

on

#FakeNews and a “Post-Fact” news cycle destroyed our sense of trust in news sources – especially in the digital space. The writers at FreePressFail courageously battle on a day-to-day basis to help the average American decipher all the lies and deceptive narratives that fly around on digital news. Today we thankfully recognize more people joining the fight.

 www.fakenewschallenge.org has taken up the torch from the writers at FreePressFail and has posted “rules” for calling out #fakenews:


Description

Post-facto fake news refers to news items or claims that are already known to be false, either by work from organizations like Snopes and Factcheck or by the general public on social media.

For this challenge, we will only consider claims that are outright false or outright true. For example, “eating fruit prevents cancer” — a truth assertion here is dodgy at best, but for headlines like “Hillary has a body double” we can be confident about truth assertion.

We will select headlines for the competition where we can be confident in asserting its veracity.

Task

Input

A claim/headline string with accompanied URL, if any.

  • “Pope endorsed Trump”
  • “Climate change is a Martian takeover operation”

Output

Four required values
  • 1. Boolean fakeness indicator
  • 2. Confidence score
  • 3. Provenance URL to support
  • 4. A confidence threshold for accepting/rejecting

Data

Training Data

We will provide the training data

Development Data

Teams will use part of the training data as a development set

Test Data

Test data is withheld

Auxiliary Data

In addition to training data, you are free to use all of the internet. The solution has to work in real-time.

Evaluation Criteria

Submissions will be evaluated based on a 0-1 loss using the submission-provided threshold. All winning entries must provide a provenance URL for their solutions.

Awards

There is a total of $2000 to be awarded for the top-5 teams.
Each team will get cash awards proportional to their accuracy.
Submissions achieving more than 10% false positive or false negative rates are not considered.
Contesting teams must be willing to open source their systems.

Timeline

Teams register by: Jan 1st, 2017
1st batch of training data released: Jan 15th, 2017
2nd batch of training data released: Feb 15th, 2017
Submissions due: June 1st, 2017

Continue Reading
Advertisement