Have you ever hesitated before pasting a confidential document into ChatGPT? Especially when it involves a business contract or sensitive information. After all, nobody wants their data to end up on a remote server. Jan.ai offers a different approach. This free and open-source application runs AI models directly on your computer. It also works with cloud services like OpenAI, Claude or GeminiBut once installed on your PC, how good is it really? And more importantly, is your machine powerful enough to take full advantage of it?
How does it work on a daily basis?
As a reminder, Jan.ai is free software released under the AGPL-3.0 license. It is based on Tauri, a lightweight framework that notably limits its memory consumption. You can install it on Windows 10 and 11 (x64), but also on macOS and Linux.
In practice, you have two ways to use it. The first is to rotate a model directly on your computerYou download an open model, then Jan.ai uses llama.cpp and the Nitro engine to run it locally. Everything then happens on your machine.
The other option is to use cloud models. Simply connect your API key to access OpenAI, Claude (Anthropic), Gemini (Google), Mistral, or Groq from within Jan.ai. This allows you to stay within the same interface, without having to switch between multiple tools.
On the data side, your conversations and metadata are recorded on your disk in standard JSON format. Therefore, you can easily view, move, or use them.
However, be aware that as soon as you use a cloud API, your requests are sent to an external server. Confidentiality then depends on the terms and conditions of the provider you use.
Which models can you use?
To run an LLM locally, Jan.ai provides access to numerous models through its Hub: Llama, Mistral, Gemma, Qwen, DeepSeek, and the Jan-v3-4b model. If you're a beginner, you'll likely encounter terms like GGUF, Q4_K_M, Q8, 7B, and 13B. Simply put, the larger a model is, the more hardware resources it requires. Compression formats reduce this load, albeit with a slight compromise in accuracy.
The material requirements are very real:
- La RAM (RAM): With 8GB of RAM, stick to smaller, compressed 7B models. For smooth, lag-free performance, aim for at least 16GB of RAM.
- The graphics card: a dedicated GPU (notably NVIDIA or Apple Silicon chip) significantly speeds up generation compared to a simple processor
- Storage: Allow between 4 and 8 GB per model downloaded to your hard drive
Jan.ai or ChatGPT, which one to choose?
If you are hesitating between Jan.ai and ChatGPT, first ask yourself the question: what do you expect from your AI? Because everything depends on your priorities.
With Jan.ai, you choose sovereignty. Your data stays on your machine, the tool works without an internet connection, and you benefit from a multitude of open engines without any subscription. In return, you must check your PC's hardware capabilities and configure your models.
With ChatGPTYou prioritize immediate simplicity. You ask your question and receive a highly complex answer without worrying about your computer's processing power. However, your exchanges are processed on remote servers, a network connection is required, and the ecosystem remains closed.
The advantages and limitations
To summarize our Jan AI review, here's what you need to remember before you get started:
What you will appreciate:
- Your data stays with you: a decisive advantage for processing professional or sensitive documents.
- 100% offline use once the model is downloaded
- Ergonomic design that eliminates the need to open a control terminal.
- Free access to local models, without recurring subscription
Limitations to anticipate:
- Local 7B models remain less efficient than large cloud models in highly sophisticated reasoning.
- Disk space is quickly used up if you test multiple files
- A short learning phase is needed to become familiar with the size and quantification of the models.
