The Automated Librarian: Part 1 - Architecting an Intelligent eBook Library in Drupal 11
January 28, 2026We've all been there: a sprawling collection of information buried deep in a file structure on your computer's hard disk or network drive. A structured folder system helps, but finding a specific concept across that information still requires manual effort. I had this exact problem with my eBook collection that I've built over the years.
Having been a Drupal developer for the better part of 20 years, I knew that I could leverage Drupal's robust architecture to build a modern, automated solution to bring my library into the modern era. I didn't just want a list of files; I wanted a searchable, AI-summarized, high-performance library.
In this series, I'll walk you through how I built an eBook Library; a custom Drupal 11 application, that was designed to ingest, analyze, and index my collection automatically using Migrations, Solr, Ollama (AI LLM), and the Open Library REST API.
This post is part of a series. Check out the full roadmap at The Automated Librarian: A Drupal 11 Data Discovery.
Expectations: What The System Does and Does Not Do
Let's start by setting some expectations. This system is not designed for public consumption and purposely running inside my local network not exposed to the Internet.
First of all, a lot of my library was purchased and therefore, I don't have the publisher rights to expose the content publicly. I don't imagine getting sued for copyright violations is a fun thing to be a part of. Second, this is purely a "lab" type project to gain more experience with the technologies involved to solve a real-world problem. Third, I wanted to do as much of this using Drupal as possible and not rely on an external script written in another language such as Python.
The Architecture: A Modern Drupal 11 Stack
- The Core: Drupal 11 with a custom Media Entity.
- The Ingestion Engine: A migration that crawls my eBooks folder on a network drive.
- The Discovery: A containerized Apache Solr instance connected via Search API for full-text search and faceted filtering.
- The Intelligence: A containerized Ollama instance to summarize the content.
- The Community: Open Library REST API integration to bridge the metadata gap.
Why AI Summaries?
When I started this project, I relied on standard metadata extraction using the Kiwilan\Ebook library. I quickly ran into the frustrating reality: most eBook files are metadata ghost towns. Even when a file has internal properties, the "Publisher," "Author," and "Keywords" fields are often empty, misspelled, or just plain wrong. Plus, all of this metadata can be different across different file formats.
I actually spent 5 hours manually fixing the metadata in my system just to get it to a baseline level of organization. This is where I asked myself if AI could help and why AI moves from being "cool" to being "essential."
By integrating AI, I move from a passive system to an active one. The AI doesn't just read the tags; it understands the content. It can synthesize a 400-page book or technical manual into a short summary and identify core themes that aren't explicitly tagged. In a world where your data is only as good as your ability to find it, AI does the grunt work; fixing the "dirty data" problem that manual extraction can't solve.
What's Coming Next?
Over the next few posts, I'm going to pull back the curtain on the code:
- Part 2: The Migration Engine: How a custom "structure-aware" process plugin and the
Kiwilan\Ebooklibrary turn raw file paths into rich metadata. - Part 3: Search and Facets: Configuring Search API and Solr for that professional "library-catalog" feel.
- Part 4: Community Input: Integrating the Open Library REST API for deeper metadata.
- Part 5: The AI Connection: Integrating Ollama to "read" the books and generate descriptions.
Next up: Architecting the Migration and the "Automated Librarian".
P.S. – I was incredibly honored to have this project featured in The Drop Times recently! It's been great seeing the community's reaction to the 'Automated Librarian' journey.
0 Comments
Login or Register to post comments.