Learning_AI

Bibliography - books, essays, sites, & learning resources

Probably around half of these are free. But if you’re serious about learning, be serious about investing something in your learning. If you’re like me, it will help you take your learning project more seriously.

Reach out if you have ideas for things to add.

Meta

These resources relate to how I’ve structured the project itself—e.g. what assumptions I’m making about how to learn.

• A.G. Sertillanges, The Intellectual Life: Its Spirits, Conditions, Methods - An old book from a scholar-priest who knew how to manage attention & curiosity in the service of a fulfilling intellectual life.
• William Zinsser, Writing to Learn - An incredible book; the title says it all. How to use writing to learn anything.
• Dreyfus model of learning & mastery
• Scott Young, Ultralearning - my notes
• Iain McGilchrist, The Master & His Emissary & The Matter with Things - I don’t think anyone understands the modern world as deeply as McGilchrist.
• Andy Hunt, Pragmatic Thinking & Learning: Refactor Your Wetware
• Cal Newport, So Good They Can’t Ignore You & Slow Productivity
• Joshua Foer, Moonwalking with Einstein - The lost arts of memorization.

Beginner - start here

• Deeplearning.ai - a range of courses from beginner through advanced.
  • AI for Everyone is a good place to start if you’re totally new.
  • Machine Learning specialization
• Wooldridge, A Brief History of Artificial Intelligence - great intro to the history and the basic concepts. Well written & readable. Worthwhile even if you aren’t going to do the coding.

Intermediate

• Andrej Karpathy’s vide “Intro to Large Language Models”
• Stephen Wolfram’s essay on how LLMs work - dense but incredibly thorough.
• 3blue1brown video course on neural networks - long explainer videos.
• Andrej Karpathy’s Github repos - he’s doing tons of projects that you can follow along with, particularly Eureka Labs, which is going to be some sort of course about AI “taught” with the assistance of AI tutors.
• Raschka, Building a Large Language Model (from Scratch) - I’m working through this one already; it’s great so far.
• Practical Deep Learning for Coders
• Howard & Gugger, Deep Learning for Coders with fastai and PyTorch

Advanced

• Practical Deep Learning for Coders, part II
• Andrej Karpathy’s vide series “Neural Networks: Zero to Hero”
• fast.ai
• GPT learning hub
• Yuan & Yuan, Open Source LLMs (Manning 3 Projects)
• Stephens, AI Algorithms (Manning 4 Projects)
• Maiden, Neo4j (Manning 3 Projects)
• Elgueta, LangChain (Manning 3 Projects)
• Vaswani et al, “Attention is all you need” - Come for the cheesy Beatles reference; stay for the groundbreaking definition of the transformer architecture
• Rothman, RAG-Driven Generative AI: Build custom retrieval augmented generation pipelines with LlamaIndex, Deep Lake, and Pinecone - [Highly recommended by Jeremy Revenel]

Taxonomies, Ontologies, and Knowledge Graphs

• Heather Hedden, The Accidental Taxonomist - be sure to purchase the third edition (or newer, if applicable), which contains a new chapter on ontology
• Bob DuCharme blog on semantic web technologies
• David Knickerbocker, 100 Days of Networks project
• David Knickerbocker, Network Science with Python -
• Lulit Tesfaye, “What is a Semantic Layer? Components and Enterprise Applications” - discusses taxonomy, ontology, kg, and other components
• Hogan et al, “Knowledge Graphs” - incredibly thorough and long introductory paper. See the 5 page appendix at the end for a history of KG technology.
• Amit Singhal, “Introducing the Knowledge Graph: things, not strings” - the foundational blog post on knowledge graphs in terms of general awareness.
• Richard J. Trudeau, Introduction to Graph Theory - published by Dover, you can get this new on Amazon very cheap. It’s a great introduction to the math side of graph theory, which underpins all the exciting work being done on knowledge graphs right now. What’s more, it’s written with style & attitude, and has tons of example problems along with suggestions for further reading.
• Dean Allemang, blog on KGs & AI - one of the KG OGs.
• OWL 2 Primer (W3C)
• Barry Smith, Ontology for Systems Engineering
• Uschold, Michael - Demystifying OWL for the Enterprise
• Allemang et al, Semantic Web for the Working Ontologist
• The Semantic Layer

The Textual Encoding Initiative

I’m particularly interested in how ontologies and knowledge graphs can be used in education. For this reason, the Textual Encoding Initiative is a compelling project.

• A very gentle introduction to the TEI markup language
• TEI and XML Markup for Absolute Beginners
• TEI Simple
• Ciotti, Tomasi, Peroni, Vitali, “An Ontology for the TEI (Simple): One Step Beyond”
• TEI by Example
• Folger Digital Texts Archive - Most of Shakespeare, in TEI Simple format.
• TEI Publisher - A great open-source publishing tool for TEI-formatted ebooks

Using KGs for AI

A subfield of the KG+AI field: using KGs to help you build more-reliable AI.

• Panagiotis Alexopoulos, “Knowledge Graphs & Large Language Models Bootcamp” - This six-hour O’Reilly course is an excellent intro to knowledge graphs and how to apply them to AI applications.
• Michael Iantosca, Helmut Nagy, and William Sandri, “Document Object Model Graph RAG: A semantic, content-first, and knowledge-management architecture for neuro-symbolic RAG” / pdf version - A clear overview of the limitations of stochastic LLMs and even RAG models, along with a clear articulation of an alternate, trustworthy model.
• Philip Rathle, “The GraphRAG Manifesto: Adding Knowledge to GenAI””
• Ashleigh Faith, Jesús Barrasa, & Dean Allemang, “Which is better for AI: Property Graph or Triple Store?”
• WhyHow.ai blog
• Unlocking LLM Power with Organizational KG Ontologies
• Ben Lorica, “Charting the Graphical Roadmap to Smarter AI””
• Ben Lorica, “GraphRAG: Design Patterns, Challenges, Recommendations” - The most comprehensive overview of different possible architectures for your KG+AI project.
• TUTORIAL: Using LangGraph and Graphiti: Building an agent with LangChain’s LangGraph and Graphiti

Using AI for Ontologies & KGs

Another subfield of the KG+AI field: using AI to help you create, manage, and query KGs.

• Sequeda, Allemang, Jacob, “A Benchmark to Understand the Role of Knowledge Graphs on Large Language Models’ Accuracy for Question Answering on Enterprise SQL Databases”