DKPro C4CorpusTools

DKPro C4CorpusTools

DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate removal, language detection, and near-duplicate removal.

License

DKPro C4CorpusTools is available as open-source software under the ASL license.

Download

Visit our GitHub page.

Primary contact

  • Dr. Ivan Habernal