How to Run a Local LLM for Novel Writing on Your Own PC: What Your Computer Needs and 5 Steps
If your PC has around 16 GB of RAM, you can run a local LLM for novel writing on your own computer with free tools. Your manuscript never leaves your machine, and you can ask for rewrites as often as you like. How well it writes, though, depends on your PC and the model you pick. This guide walks you through checking whether your PC can handle it, all the way to getting a full chapter written.

Key takeaways
- With a PC that has about 16 GB of RAM, you can try a local LLM for novel writing using free tools such as LM Studio or Ollama
- Check your RAM and your GPU's dedicated memory first. Start with a small model and compare how natural the writing reads
- The amount the model can keep in mind (context length) is short, so pass your story bible and a summary of the previous scene every time, and write one scene at a time
- If you plan to publish or sell, check each model's terms of use (license) on the page for the exact version you use
What it means to run a local LLM for novel writing
Running AI locally means installing a text-generating AI (a large language model, or LLM) on your own PC and having it write your novel entirely inside that PC. Browser-based AI such as ChatGPT or Claude sends what you type to the company's servers and returns an answer. With a local setup, your own PC does that computing.
The short answer: if your PC has around 16 GB of RAM, you can start trying it today with free tools. How well it writes, however, varies a lot with your PC and the model you choose. This guide first checks whether your PC can handle it, then installs the tools, and goes all the way to getting a chapter written.
People choose local AI for roughly three reasons:
- Your writing stays on your machine: you can write manuscripts nobody has seen yet, or subjects you would rather not share, without sending them to outside servers
- No usage limits to worry about: however many rewrites you ask for, none of them count against a service's usage cap (they do cost electricity and PC time)
- You can write without the internet: some tools keep working where there is no signal, once you have downloaded a model

In exchange, there are three trade-offs: your PC needs a certain level of performance, you have to install the tools and models yourself, and writing quality in your language depends on the model, so the prose may not come out as natural as you hoped. One easy way to see the difference: cloud AI is a service where the company takes on these three burdens for you.
First, check whether your PC can run it
The first thing to check is your memory (RAM). LM Studio, a tool you operate through an app window, recommends 16 GB or more on both Windows and Mac in its official system requirements. On Mac, it supports Apple silicon (listed officially as M1 to M4); Intel Macs are not supported for now. On Windows, it also recommends at least 4 GB of dedicated memory (VRAM) on the graphics processor (GPU) (checked on the official site on November 15, 2026).
On Windows, you can see how much RAM you have under Settings → System → About. Your GPU's dedicated memory is shown in Task Manager under Performance → GPU.
Macs share memory between the CPU and GPU, so skip the GPU question and go by your total memory alone.
Also look at the model's "size." Numbers such as "4B," "12B" or "31B" in a model's description are the number of internal parts (parameters). Bigger models tend to write better but use more memory. For example, Google's official Gemma page gives the following memory guidelines for loading versions compressed to 4 bits.
| Size | Approx. for 4-bit compressed version | What PC it suits |
|---|---|---|
| Small (about 2B–4B) | About 3–5 GB | Easy to try even on a laptop with 16 GB of RAM |
| Medium (about 12B) | About 7 GB | A GPU with plenty of dedicated memory, or a Mac with memory to spare |
| Large (about 26B–31B) | About 14–18 GB | A GPU with a large amount of dedicated memory, or a Mac with lots of memory |
These figures cover only the model itself. The official page also notes that the longer a conversation gets, the more additional memory is needed to keep it in mind. The figures also vary by version and tool, so check the model's page before you install it.
Two kinds of tools: an app with a window, or a command-line tool
There are many tools for running models locally. The two that beginners use most are below. With both, you can download models from inside the tool.
| LM Studio | Ollama | |
|---|---|---|
| How you use it | Pick a model in the app window and chat with it | Can be started from an app, but mainly used through commands |
| Supported PCs | Windows, Mac (Apple silicon), Linux | Windows, Mac, Linux |
| Does anything leave your PC? | States that while you use local models, your chats and documents are not sent outside your PC | States that while running locally, the company does not see your prompts or data |
| Without internet | States that everything runs offline once you have downloaded a model | You need internet to download models |
| Watch out for | There are also paid cloud features. Check pricing on the official site | You can also choose models that run in the cloud. To stay on your PC only, pick a local model |
Both also offer cloud features that run on the company's servers. If your goal is to keep everything on your machine, it matters to check whether the model you chose is a local one before you use it. Ollama also has a setting to turn off cloud features and use it locally only.
5 steps to getting a chapter written
Once you have chosen a tool, go in the order below. The order matters for steps 4 and 5: with the default settings, a local model may not be able to keep long text in mind, so you set the length before you start writing.
- 1Install the tool from its official page
- 2Download one small model and test that it runs
- 3Have it write a short scene in English and check how natural it reads
- 4Set how much it can keep in mind (context length)
- 5Give it your story bible first, then write one scene at a time
Steps 1 and 2 are about confirming that a small model "runs." Ollama's official quickstart also uses the smallest version of a model as its example, and even that is a download of about 7 GB. The "about 3–5 GB" in the table above is the amount for loading a 4-bit compressed version, so it differs from the version the tool downloads. Check your free disk space first.
In step 3, don't ask for a long chapter right away. Have it write a short scene, such as "Two people meet for the first time in 10 years at a station in the rain, in about 200 words." Give the same request to two or three models and compare them, and you will quickly see which handles your language well. If the writing feels stiff here, it is faster to switch to a larger model, or simply decide to write that story with a cloud AI.
In step 4, "how much it can keep in mind" is the amount of text the AI can read at once. According to Ollama's official documentation, if your GPU has less than 24 GiB (about 26 GB) of dedicated memory, the default is 4k (about 4,000 tokens; a token is the unit AI uses to count text). The longer you set it, the more memory it uses, so raise it gradually to suit your PC.
In step 5, paste in a story bible that summarizes your characters and rules first, then have it write one scene at a time. The next section explains why.
Prompt example for a local AI
When you write a long novel with a local model, the biggest problem is that it "forgets what was decided earlier." When the context length is short, the names and settings you gave at the start get pushed out of what it can read as the conversation grows.
To get a sense of length, we counted the Japanese chapters of stories published on Original Novel: the median chapter was 3,806 Japanese characters (about 1,900 words), and half of all chapters fell between 3,215 and 4,126 characters (about 1,600–2,100 words) (227 chapters, November 14, 2026). A satisfying chapter is a couple of thousand words on its own. Add a story bible and a synopsis of the previous chapter on top of the new chapter, and the default length may not be enough.
So keep every prompt down to three parts: "a short story bible + a summary of the previous scene + the one scene to write now." Replace the parts in [brackets] and use it.
You are a novelist writing in English. Follow the story bible below and write only one scene. [Story bible] - Main character: [name] (age / catchphrase "[...]" / how they refer to themselves) - Other character: [name] (relationship to the main character) - Setting: [era, place, season] - Must not change: [...] [Summary of the previous scene (up to 3 lines)] [...] [Scene to write now] A scene in which [character] [does something]. About 400 words. Use plenty of dialogue, and end on a line that makes the reader want to know what happens next. Do not add names or events that are not in the story bible.
Paste each finished scene into a text file yourself, building it up as you go. The trick is to pass a summary, not the full text, when you ask for the next scene. When a chapter is done, have the AI sum it up in three lines and add that to your story bible as "settled facts," and later chapters are less likely to contradict earlier ones.
Before publishing your novel, check the model's terms of use
Even if a local model is free to download, the conditions for using it (the license) differ from model to model. If you are only writing for your own enjoyment, there is less to worry about. But if you plan to post on a writing site or sell your work as a book, check these three things on the model's page:
- Whether commercial use is allowed: some use common, widely permissive licenses, while others add their own conditions
- Whether there are attribution rules: some require you to show the name or include a copy of the terms when you distribute the model itself or something built with it
- Prohibited uses: some set a separate "acceptable use policy" and make following it a condition
For example, Google's Gemma 4 lists Apache 2.0, a common license, on its distribution page. Meta's Llama 3.1 uses its own license, which requires things such as following its acceptable use policy and displaying "Built with Llama" when you distribute the model. Separately from Gemma 4, the Gemma Terms of Use, which apply to earlier versions and others, state that Google claims no rights to outputs you generate with Gemma. Terms change from version to version, so check the page for the version you use. Copyright of AI-written text and each writing site's rules on AI exist separately from the model's terms.
Local AI or a browser-based site: when to use which
The strength of local AI is that you can rewrite as many times as you like without your manuscript leaving your machine. For people who want to keep extending their own writing bit by bit, or who want to write while traveling without internet, it offers something nothing else can.
On the other hand, you have to put everything together yourself, from managing your story bible to illustrations and translation into other languages. Original Novel is a browser-based site that takes care of that part. The "pass the story bible every time" step you did by hand with the prompt example in this guide is handled behind the scenes for each story, and before each chapter is published, AI checks it for contradictions with earlier chapters and mistakes in names.
You can also split the work: manuscripts you want to rewrite freely stay local, and stories you want others to read, with the settings managed for you, go on the site. If most of your stories are ones you don't want to leave your machine, try local first; if you want something readable right away, try the site first. Either way, you will save effort.
Read stories written by AI
See more: Read stories written by AI ›Stories on this site that AI wrote while following a story bible. Sorted by how often they are read.
Sources (date checked)
- System RequirementsLM Studio / Checked Nov 18, 2026
- LM Studio Privacy PolicyLM Studio / Checked Nov 18, 2026
- QuickstartOllama / Checked Nov 18, 2026
- Local FAQOllama / Checked Nov 18, 2026
- Context lengthOllama / Checked Nov 18, 2026
- Gemma 4 model overviewGoogle AI for Developers / Checked Nov 18, 2026
- Gemma Terms of UseGoogle AI for Developers / Checked Nov 18, 2026
- Llama 3.1 Community License AgreementMeta / Checked Nov 18, 2026
FAQ
Is running a local LLM free?
LM Studio includes running models locally in its free tier, and Ollama is released under the MIT License. Many models can also be downloaded for free. However, both tools also have paid cloud features, and prices and terms can change, so check their official pages.
Can I write fiction on a laptop without a GPU?
Ollama's official documentation says it runs on the CPU when no NVIDIA GPU is found on Linux, and it also describes a setting for CPU-only use. That said, LM Studio recommends at least 4 GB of dedicated GPU memory on Windows, and how fast text comes out depends on your PC. The safest approach is to have the smallest model write a short scene first, check the speed and how natural it reads, and then decide.
Can a local LLM run on a smartphone?
Some models are built small for smartphones, and Google's official Gemma page also lists memory guidelines for its mobile versions. For writing a long novel, though, a PC is better suited in terms of context length and speed. If you try it on a phone, start with drafts of short scenes.
Is fiction written by a local LLM lower quality than ChatGPT's?
It varies a lot with the model's size and how much of your language it has learned. Models small enough to run on a home PC can produce less natural prose than large cloud models. The most reliable way is to give both the same request and pick whichever suits your story.
Does it work with non-NVIDIA GPUs (such as AMD)?
In some cases, yes. Ollama's official documentation says it can accelerate on NVIDIA GPUs (compute capability 5.0 or higher), on AMD GPUs through ROCm v7 or Vulkan, and on Macs through Apple's Metal. The supported models and driver versions are specified in detail, so check that your GPU is on the list on the official page.




