Automated fact-checking is becoming a key tool in the fight against disinformation. In this article, I will show you how to combine n8n, RSS feeds, and LLM models to create an effective content analysis workflow – step by step, taking into account limitations and best practices.
Why is fact-checking automation necessary?
Disinformation is spreading faster than ever – according to the Edelman Trust Barometer 2023 report, 76% of respondents fear that fake news is being used as a weapon. Manual fact-checking cannot keep up with the scale of the problem, which is why tools like n8n are gaining importance. They allow for the automation of parts of the process, saving time and resources.
However, automation will not replace human judgment – it can only support it. It is crucial to understand where the capabilities of algorithms end and the necessity for expert intervention begins.
Key n8n features useful for fact-checking
Content parsing nodes
n8n offers several nodes that are essential for fetching and processing content:
- RSS Feed: Allows for the automatic fetching of new entries from RSS feeds. Ideal for monitoring news sites or aggregators like Gurify. This node is simple to configure – just provide the RSS feed URL and set the refresh frequency.
- HTTP Request: Enables fetching data from APIs or scraping websites. Useful when we want to access content that is not available via RSS. It is worth remembering the legal limitations related to scraping (e.g., website terms of service).
- Webhook: Allows for receiving data in real-time, e.g., alerts from external systems. It can be used for integration with tools like Slack or Discord, where users report suspicious content.
- HTML Extract: Extracts data from HTML code, e.g., article content. Useful when we want to analyze specific fragments of text rather than the entire page.
All these nodes can be combined into a workflow to create a comprehensive system for monitoring and analyzing content.
Integration with LLM models
Large Language Models (LLMs) can significantly support the fact-checking process by analyzing content for potential disinformation. n8n offers nodes for integration with popular models:
- openai: The
OpenAInode allows for the use of models like GPT-3.5 or GPT-4 for text analysis. It can be used, for example, to detect phrasing that suggests disinformation or to generate article summaries. - Hugging Face: The
HuggingFacenode enables the use of NLP models available on the Hugging Face platform. For example, you can use a model for text classification (e.g., "true" vs "false").
An example prompt for openai that can help in detecting disinformation:
"Sprawdź, czy poniższe zdanie zawiera potencjalną dezinformację. Odpowiedz 'TAK' lub 'NIE' i uzasadnij krótko:
'Nowy raport WHO potwierdza, że 5G powoduje raka.'"However, it is worth remembering that LLMs are not infallible. They can make mistakes, especially in the case of languages with fewer training data points compared to English.
n8n limitations
Despite its advantages, n8n has several limitations that are worth considering:
- Processing large data volumes: n8n is not optimized for processing hundreds of thousands of posts per minute. In such cases, dedicated solutions like Apache Kafka are better suited.
- Open-source version limits: The free version of n8n (self-hosted) has no limits on the number of workflow executions, but the n8n.Cloud version has limits (e.g., 2,000 executions/month in the free tier).
- Workflow complexity: Highly complex workflows can be difficult to maintain and debug. It is worth breaking them down into smaller, modular parts.
External tools for integration with n8n
Fact-checking APIs
There are several APIs and databases that can be integrated with n8n for automated fact-checking:
- Google Fact Check Tools API: Allows searching for labeled facts in the Google database. Requires an API key and authorization. Example use case: Checking if a piece of information has already been verified by fact-checkers.
- claimreview: A metadata standard for labeling verified content. It can be used in combination with the
HTTP Requestnode in n8n to check if specific content has already been verified. - Full Fact API: A British fact-checking service offers a limited API for partners. It is worth checking if similar solutions are available in your region.
Unfortunately, not all fact-checking services offer a public API. In such cases, you might consider scraping their sites (while adhering to robots.txt policies and terms of service).
Open-source content analysis tools
In addition to commercial APIs, there are also open-source tools that can be integrated with n8n:
- ClaimBuster: A tool for detecting sentences that require verification. It can be integrated with n8n via API. The repository on GitHub is regularly updated, which indicates active project development.
- FakeNewsNet: A collection of data and tools for detecting fake news. It can be used to train your own models or analyze content.
- Hoaxy: A tool for visualizing the spread of disinformation. It can be useful for monitoring how a piece of information spreads across the web.
Integration with RSS feeds (Gurify)
Gurify is a tool for aggregating RSS feeds from various sources. It can be integrated with n8n using the RSS Feed node. An example workflow might look like this:
- Fetching content from a Gurify RSS feed (e.g., from news sites).
- Filtering content (e.g., only those containing keywords: "election", "pandemic").
- LLM analysis (e.g., openai or Hugging Face) for potential disinformation.
- Verification via Google Fact Check Tools API or claimreview.
- Saving results to Google Sheets or sending an email alert.
Building a workflow in n8n – step by step
Typical fact-checking workflow
Below is an example workflow that can be built in n8n:
- Fetching content:
- Using the
RSS Feednode to fetch the latest entries from an RSS feed (e.g., from Gurify). - Alternatively: Using the
HTTP Requestnode to fetch data from a news site's API.
- Using the
- Filtering and data preparation:
- Using the
IFnode to filter content (e.g., only those containing keywords: "vaccines", "climate"). - Using the
Setnode to normalize data (e.g., removing HTML tags, converting to lowercase).
- Using the
- LLM analysis:
- Using the
OpenAIorHuggingFacenode for content analysis. - Example task for an LLM: "Does the following sentence contain potential disinformation? Answer 'YES' or 'NO' and provide a brief justification."
- Using the
- Source verification:
- Using the
HTTP Requestnode to check against the Google Fact Check Tools API or ClaimReview. - Example: Sending a query to the API with a phrase from the article and checking if it has already been verified.
- Using the
- Alert or saving results:
- Using the
Emailnode (e.g., Gmail) to send an alert about potential disinformation. - Using the
Google SheetsorAirtablenode to save analysis results (article title, source, LLM analysis result, link to verification).
- Using the
Ready-made templates and optimization
n8n offers several ready-made workflow templates that can be adapted for fact-checking:
However, there are no ready-made templates dedicated specifically to fact-checking, so it is worth building your own workflow based on existing ones.
Workflow optimization may include:
- Caching: Using the
Cachenode to store API results (e.g., Google Fact Check Tools) to avoid excessive queries. - Error Handling: Using the
Error Triggernode to handle errors (e.g., no response from API). - Testing: Using "Test" mode in n8n to check the correctness of the workflow before deployment.
Challenges and limitations of automated verification
Analysis errors
Automated fact-checking is not error-free. The most common problems are:
- False positives: An LLM might flag content as disinformation that is true but phrased in a sensationalist way. Example: An article about "vaccines causing side effects" might be flagged as disinformation even if it describes rare cases.
- Lack of context: Automated analysis does not take into account cultural or historical context. Example: The phrase "The Earth is flat" might be used in a satirical article, not as disinformation.
- Language: LLM models handle English better than other languages, which can lead to errors in analysis for non-English content.
Ethical and legal aspects
Automated fact-checking involves several ethical and legal challenges:
- GDPR: Processing user content (e.g., comments) requires consent or anonymization. It is worth ensuring that the workflow does not violate personal data protection regulations.
- Copyright: Scraping content from news sites may violate their terms of service. Before starting automated content fetching, it is worth checking if it is permitted.
Cases of ineffectiveness
Automated fact-checking does not work in all situations:
- Deepfake: Automated tools have difficulty detecting video or audio deepfakes. This requires advanced models for multimedia analysis.
- Fast-moving events: Real-time fact-checking (e.g., during a crisis) can be delayed because algorithms need time to analyze.
- Linguistic manipulation: Disinformation can be hidden in seemingly neutral phrasing (e.g., "some experts claim..."). Such cases are difficult for algorithms to detect.
Alternatives to n8n
Other no-code/low-code tools
n8n is not the only automation tool. Here are some alternatives:
- Zapier:
- Pros: Ease of use, large number of integrations.
- Cons: Limited advanced logic capabilities (e.g., no loops), higher costs at large volumes.
- Make (formerly Integromat):
- Pros: Visual builder, advanced scenarios.
- Cons: Less flexible than n8n for custom integrations.
- Pipedream:
- Pros: JavaScript-based, flexible.
- Cons: Requires basic programming knowledge.
Dedicated fact-checking platforms
There are also platforms created specifically for fact-checking:
- NewsGuard: Offers media credibility ratings, but lacks a public API. Can be used as a source of information for fact-checkers.
- Logically: A platform for detecting disinformation with an API for partners. It is worth checking if it offers integration with n8n.
- ClaimReview: A metadata standard, but requires manual labeling of content by fact-checkers.
Comparison of n8n with alternatives
| Tool | Flexibility | LLM Integrations | Cost | Technical Requirements |
|---|---|---|---|---|
| n8n | High | Yes | Free (open-source) | Medium |
| Zapier | Low | Limited | Paid | Low |
| Make | Medium | Yes | Paid | Low |
| Pipedream | High | Yes | Free (with limits) | High (JS) |
The choice of tool depends on specific needs. n8n works well where flexibility and LLM integration are needed, while Zapier or Make might be better for simpler tasks.
Costs and technical requirements
n8n costs
n8n is available in two versions:
- Open-source version: Free, but requires self-hosting (e.g., on a VPS). Ideal for users who want full control over their workflow.
- n8n.cloud: Paid hosting with a limit of 2,000 workflow executions/month in the free tier. Pricing:
- Startup Plan: $20/month (10,000 executions).
- Pro Plan: $50/month (50,000 executions).
Costs of external API integrations
Integration with external APIs involves additional costs:
- openai API:
- Model
gpt-3.5-turbo: $0.002 per 1,000 tokens (approx. 750 words). - Example: Analyzing 1,000 articles (avg. 500 words each) would cost approx. $1.33.
- Model
- Google Fact Check Tools API: Free, but with a query limit (undocumented).
- Hugging Face Inference API: Free for small volumes, paid for larger ones.
Technical requirements
To run a workflow in n8n, you need:
- Server:
- Minimum requirements: 1 vCPU, 1 GB RAM (for small workflows).
- Recommended: 2 vCPU, 4 GB RAM (for workflows with LLMs and large data volumes).
- API:
- API keys for OpenAI, Google Fact Check Tools, etc.
- Query limit: Monitor usage to avoid blocks.
Educational resources and communities
Documentation and tutorials
If you want to deepen your knowledge about n8n, it is worth using the following resources:
- Official n8n documentation: docs.n8n.IO – contains detailed descriptions of all nodes and usage examples.
- Video course: n8n YouTube Channel – regularly updated tutorials and webinars.
- Example workflows: n8n Workflows – ready-made templates that can be adapted.
Communities
It is worth joining communities focused on n8n and fact-checking automation:
- n8n Forum: community.n8n.IO – active discussions and technical support.
- Discord: n8n Discord – live chat with users and developers.
- github: n8n GitHub – reporting bugs and feature requests.
Conferences and webinars
If you are interested in the topic of automation and fact-checking, it is worth participating in the following events:
- No-Code Conf: An annual conference dedicated to no-code tools, including n8n. nocodeconf.com.
- Fact-Checking Summit: A conference dedicated to fact-checking. globalfact.org.
Participating in such events allows for the exchange of experiences and learning about the latest trends in the industry.
Summary
Automated fact-checking using n8n is a powerful tool that can significantly improve the work of fact-checkers. Thanks to integration with RSS feeds, LLM models, and external APIs, you can build a workflow that monitors, analyzes, and verifies content in an automated way.
However, automation has its limitations. LLM models can make mistakes, and algorithms will not replace human judgment in complex cases. Therefore, it is crucial to treat such systems as support, not a replacement for traditional fact-checking.
If you want to start your adventure with automated fact-checking, n8n is a great choice – flexible, free, and easy to use. However, it is worth remembering to optimize your workflow and monitor its performance to avoid problems related to large data volumes.
I encourage you to experiment with n8n and share your experiences in communities focused on this tool. Fact-checking automation is just the beginning – the future will likely bring even more innovations in this field.
Automation will not replace human judgment, but it can significantly improve it. The key is finding a balance between technology and expert knowledge.
If you are interested in the topic of automating the fight against disinformation, I also recommend my previous post: Automating the fight against disinformation: How to combine n8n, RSS, and fact-checking?.
Comments