Verifis

quick verification and explanation using LLM bundled in a Chrome extension

Beginnings

While noticing how artificial intelligence is increasingly writing content online that I personally consume, specifically through LinkedIn/X and LLM outputs, I assumed its use will only continue to grow, eventually reaching blogs, news, and other sources of information where verifiability of claims is important. This sparked the motivation to create some piece of software that could help seek the truth behind statements that may seem untrue.

The Problem

The big question here is: how can we verify claims within text written by AI?

The problem is that "verification" itself is ultimately subjective, raising the question of who will verify the verification.

Technical Details

Drawing inspiration from the success of the RLAIF (Reinforcement Learning from AI Feedback) method of using an LLM judge to evaluate output quality, I decided to use a secondary LLM as the fact-checker which would reference credible sites to form its opinion and link them for the user to conduct further research.

The container I settled on was a browser extension that would automatically fact-check highlighted text as an overlay for minimal friction for the user, so users wouldn't have to switch away from the current page or even move their cursor to open up the extension. I figured an API call to a top LLM provider would be the best route for the brains, settling on using Gemini because of Google's free 1 year of Gemini Pro access for students, but I did consider other LLM providers as well[1].

I later noticed the same act of highlighting text and using it as input would well serve as an explaining bot as well, as I've often found myself blanking on terms I'm not familiar with when reading certain articles or research papers, so I added a toggle to swap between "fact-check" and "explain" mode quickly.

Further Thoughts

The main point of this project was to quickly prototype an MVP of my idea and get a feel for the process of building.

Some ideas I have for future changes:

  • Local model to remove API costs or swap LLM provider
  • Better prompt engineering
  • Automatically gains context from webpage
  • Better curated credible sources

Notes

[1] I think Grok is best for its mantra of "truthseeking AI" and generally more objective viewpoints.