DeepSeek and Ollama: Running LLMs locally with the power of your GPU
The AI and LLMs development universe is shifting rapidly. Just last week, Nvidia, the powerhouse behind AI hardware saw its stock going down, Why you ask? Because the world is realizing that you don’t need to rely on massive cloud infrastructure or expensive GPUs to harness the power of AI.Deepseek, the new model deveolped by the chinese DeepSeek Inc offered an alternative to models already consolidated in the market.But let's forget the volatility of the stock market and learn how we can run a local instance of Deepseek in our machines step by step.
Step 1: Install Ollama
First, you will need to install Ollama on your machine. Open your terminal and run the following command:$ curl -fsSL https://ollama.com/install.sh | shThis script will set up Ollama on your system. But what is Ollama David?Basically, Ollama is an open-source tool that runs large language models (LLMs) directly on a local machine using the power of your GPU.
Step 2: Verify the Installation
Once installed, check if Ollama is working by running:
$ ollama
If you see a response, ollama was installed successfully.
Step 3: Choose Your DeepSeek Model
Ollama supports multiple versions of DeepSeek. The two main models available are:
Parameters: 7B (7 billion parameters).
Best for: Beginners, lightweight tasks, and users with limited hardware.
Use cases: Simple text generation, basic coding assistance, and experimentation.
Hardware requirements: Can run on CPUs or low-end GPUs.
Parameters: 70B (70 billion parameters).
Best for: Advanced users, complex tasks, and professional use cases.
Use cases: Advanced coding, research, creative writing, and large-scale data analysis.
Hardware requirements: Requires a high-end GPU for optimal performance.
What Are Parameters?
Parameters are the building blocks of AI models. They determine how much knowledge and complexity the model can handle. Here’s a quick breakdown:
7B (7 billion parameters): Smaller models like DeepSeek-R1 are faster and require less computational power. They’re great for basic tasks and experimentation.
70B (70 billion parameters): Larger models like DeepSeek-V3 are more powerful and capable of handling complex tasks with higher accuracy. However, they require more resources to run.
Step 4: Run Your Chosen Model
Once you’ve decided on a model, you can run it using Ollama. For example:
To run DeepSeek-R1:
$ ollama run deepseek-r1
To run DeepSeek-V3:
$ ollama run deepseek-v3
Step 5: Start Experimenting!
Now that your model is running, you can use it for:
Coding assistance: Generate code snippets or debug errors.
Creative writing: Write stories, poems, or scripts.
Research: Analyze data or generate insights.
Experiment with parameters: Adjust the model’s settings to optimize performance for your specific use case. Ollama has a lot of models to explore.
Conclusion
Whether you’re a beginner or an advanced user, running DeepSeek locally with Ollama opens up a world of possibilities. Start with DeepSeek-R1 for lightweight tasks or go all-in with DeepSeek-V3 for advanced projects.