This is a wall of text but this is the result of a custom modfile in ollama. Modfile: FROM llama3.2:3b-instruct-q4_0 PARAMETER temperature 0.1 PARAMETER num_ctx 8192 SYSTEM You are a book editor whose only job is to take in raw text and add correct punctuation. Do not add or remove any words. If there is punctuation that is incorrect, you may remove it or move it to the appropriate position within the text. My prompt: (base) jjones@Jamess-MacBook-Pro ollama % ollama run llama3.2:3b-instruct-q4_0-8k >>> 0:00 have you ever set a goal like a big goal 0:02 for the year only to lose momentum in a 0:04 few weeks well in today's video I'm 0:06 going to show you exactly how to use the 0:07 full focused planner to set powerful 0:10 quarterly goals that stick we're going 0:12 to talk about habit goals versus 0:14 achievement goals and why your key 0:15 motivation matters more than you think 0:18 hey there if you're new here I'm Riley 0:19 I'm an Air Force officer and a giant 0:21 productivity nerd if you're ready to 0:22 start setting intentional achievable 0:24 goals that you actually stick to this is 0:27 the video for you so why quarterly I get 0:31 it it sounds kind of corporate kind of 0:33 boring but there's a reason that 0:34 successful businesses actually break 0:36 down the results by quarter LLM Output: Here's the edited text with correct punctuation: 0:00 - Have you ever set a goal, like a big goal? 0:02 For the year, only to lose momentum in a few weeks. 0:04 Well, in today's video, I'm going to show you exactly how to use the full-focused planner to set powerful quarterly goals that stick. 0:06 We're going to talk about habit goals versus achievement goals and why your key motivation matters more than you think. 0:12 Hey there! If you're new here, I'm Riley. 0:19 I'm an Air Force officer and a giant productivity nerd. 0:21 If you're ready to start setting intentional, achievable goals that you actually stick to, this is the video for you. 0:27 So, why quarterly? It sounds kind of

I won't post another wall of text but I did get the basic implementation of a semantic chunker working, although it is slow on my macbook pro. Need to try it out on my desktop pc tomorrow to see if I can gain a little speed on the gtx1080 and 32gb or ram. "messages": [ "Processed 59 sentences into 13 chunks for video ID io_LVnhtRVc." ]

@Paraskevi Kivroglou I am thinking of breaking agents into subagents that each do a specific part of the system in the video. I'm not super clear on the vision yet.

I have not gotten to the point where I want to try and host Ollama on an external server. In my limited understanding, it's going to be pretty expensive to try and rent out the hardware you would need to allow ollama to run at any type of acceptable speed for real use cases. I'm interested in seeing what others have to say about this. To reference a locally running ollama instance, you would use the OpenAI python package if writing your own code, or the OpenAI Chat package in langchain, where you would just change the host to your local ollama instance. This is a concise article on using the ollama api with python. https://dev.to/jayantaadhikary/using-the-ollama-api-to-run-llms-and-generate-responses-locally-18b7

@Carl Scutt What computer are you using? I have a 2019 macbook pro with 16gb RAM and it is enough to run ollama for embeddings and the llama3.23b model fairly fast. It's enough for learning.