Choosing the Right Tool for the Job
A decision framework for picking the right category of AI tool — provider SDK, orchestration framework, vector database, or local runner — for a given problem.
Introduction
With dozens of tools ahead in this course, the most valuable skill is not memorizing all of them — it is quickly narrowing down which category solves the problem in front of you. This lesson gives you that filter before the category-by-category lessons begin.
A Real-Life Analogy First
Picture a home toolbox: a hammer, a screwdriver, a wrench, a tape measure, and a level. A beginner handyperson might try to use whichever tool is closest, or the one they are most comfortable with, regardless of the job. An experienced one glances at the task first — "this is a screw, not a nail" — and only then reaches for the matching tool. The tool never changes the job; understanding the job correctly is what determines which tool is even worth picking up.
Every category in this course's ecosystem map (lesson 1) is one tool in that toolbox. A provider SDK is the screwdriver — simple, direct, right for a huge number of jobs. An orchestration framework is more like a full power drill with attachments — more capable, but overkill for driving in a single screw. This lesson is the "look at the job first" habit, applied to AI tooling.
Start With the Problem, Not the Tool
A common mistake is picking a trendy framework first and then trying to force a problem to fit it. The more reliable approach is to state the problem in plain language, then match it to the category of tool built for exactly that problem.
"I just need the model to answer a question"
A provider SDK call is enough — no framework needed at all.
"I need the model to use my own data"
Retrieval: embeddings, a vector database, and a RAG pipeline.
"I need multiple steps chained together"
An orchestration framework like LangChain or LlamaIndex.
"I need multiple AI 'roles' collaborating"
An agent framework like CrewAI or AutoGen.
A Decision Table
| If you need to... | Reach for... | Covered in |
|---|---|---|
| Call a hosted model directly | A provider SDK (openai, anthropic) | Lessons 5–6 |
| Answer questions from your own documents | Embeddings + a vector database + RAG | Lessons 12–15 |
| Chain multiple prompts/tools together | LangChain or LlamaIndex | Lessons 9–10 |
| Coordinate multiple collaborating agents | CrewAI or AutoGen | Lesson 11 |
| Adapt an open model to your own data | Fine-tuning with PEFT/LoRA | Lesson 17 |
| Avoid sending data to a third-party API | Local inference (Ollama, vLLM) | Lessons 18–19 |
| Test and monitor prompt quality over time | Evaluation & observability tooling | Lessons 16, 23 |
Worked Example
Take the earlier example: "add a chatbot that can answer questions from our documentation." Running it through the table above: it is not a single API call (the model does not already know your docs), it needs your own data (retrieval), and it likely does not need multiple collaborating agents. That narrows the answer to "embeddings + a vector database + RAG," pointing straight at lessons 12 through 15 — without evaluating every framework in this course first.
A Second Worked Example, From Daily Life
Suppose a friend, not a company, asks for help: "Can you build me something that reads my grandmother's handwritten recipe cards out loud in my own voice, so she can listen instead of read them?" Walking through the same filter: this needs to read an image of handwriting (a multimodal or OCR-capable model, touched on in lesson 22), and it needs to speak the result back in a specific voice (text-to-speech, also lesson 22). It does not need retrieval, orchestration, or agents at all — this is a case where the "simplest tool that solves the actual problem" is two provider-level capabilities chained together, nothing more.
This second example matters because it shows the filter working just as well for a small personal project as for a company feature — the categories in this course are not only for "serious" production software.
Common Mistakes
- Reaching for an orchestration framework when a single provider SDK call would do the job.
- Building a custom agent system before confirming a simpler RAG pipeline would not have solved the problem.
- Choosing a tool based on GitHub star count alone rather than whether it fits the category your problem actually falls into.
Best Practices
- State your problem in one plain-language sentence before opening any documentation.
- Start with the simplest tool in the matching category and add complexity only when you hit a real limitation.
- Revisit this table any time a new project starts — it is meant to be reused, not memorized.
Frequently Asked Questions
That is common — a production chatbot often combines a provider SDK, retrieval, and observability tooling together. Start with whichever category is the bottleneck right now.
No — frameworks earn their complexity once you have multiple steps, tools, or memory to manage. For a single call-and-respond feature, a raw SDK call is usually simpler and easier to debug.
Try prompting and retrieval first — they solve the vast majority of real problems. Fine-tuning (lesson 17) is worth the added complexity mainly for narrow, high-volume tasks where prompting alone is not accurate or consistent enough.
Yes: for your first few projects, just ask "does the model need to know something it wasn't trained on?" If yes, you need retrieval. Otherwise start with a single provider SDK call and only add a category from the table once you hit an actual wall — you will learn the rest of this framework naturally through real friction, which sticks better than memorizing it upfront.
Key Takeaways
- Start from the problem stated in plain language, not from a favorite tool.
- Most problems map cleanly onto one of a handful of categories: provider SDK, retrieval, orchestration, agents, fine-tuning, or local inference.
- Start simple — a single API call is often enough before reaching for a framework.
- This decision table works the same for a small personal project as for a company product.
- This decision table is meant to be reused for every new project in this ecosystem.
Summary
With this decision framework in hand, the rest of the course becomes a reference you can navigate directly to the category you need, rather than a list to memorize front to back.
- You can map a plain-language problem to the right tool category.
- You have a reusable decision table for future projects, tested on both a company scenario and a personal one.
- You are ready to meet the actual provider SDKs in the next lesson.