Buckyverse: Elixir Static Site Generator with LaTeX → MathML

Project Overview

Build a static site generator in Elixir that converts Markdown content with LaTeX math into HTML with native MathML. Designed for hosting digitized out-of-print books, specifically R. Buckminster Fuller's works.

Technical Stack

• Elixir - Core language
• NimbleParsec - LaTeX math parser
• MathML - Native browser math rendering (no JavaScript)
• tex4ht - Fallback for complex LaTeX (optional)
• Static HTML output - Deploy anywhere (Netlify, Cloudflare Pages, S3)

Project Structure

buckyverse/
├── lib/
│   ├── buckyverse/
│   │   ├── application.ex
│   │   ├── library.ex              # Content loading
│   │   ├── library/
│   │   │   ├── book.ex             # Book struct
│   │   │   ├── chapter.ex          # Chapter struct
│   │   │   └── page.ex             # Page struct
│   │   ├── latex/
│   │   │   ├── parser.ex           # NimbleParsec LaTeX parser
│   │   │   ├── mathml.ex           # AST → MathML renderer
│   │   │   └── tex4ht.ex           # Fallback to tex4ht
│   │   ├── markdown/
│   │   │   └── renderer.ex         # MD + LaTeX → HTML
│   │   ├── templates/
│   │   │   ├── layout.ex           # HTML layout
│   │   │   ├── page.ex             # Page template
│   │   │   ├── home.ex             # Home page template
│   │   │   └── components.ex       # Reusable components
│   │   ├── build.ex                # Static site generator
│   │   └── watch.ex                # Dev server with live reload
│   └── buckyverse.ex
├── priv/
│   ├── books/                      # Content directory
│   │   └── synergetics/
│   │       ├── _book.yaml          # Book metadata
│   │       ├── 100-universe/
│   │       │   ├── _chapter.yaml   # Chapter metadata
│   │       │   ├── 00-index.md
│   │       │   └── 01-universe.md
│   │       └── figures/
│   │           └── fig-100.01.svg
│   └── static/
│       ├── css/
│       │   └── style.css
│       └── js/
│           └── app.js
├── _site/                          # Build output
├── mix.exs
└── README.md

Phase 1: Project Setup

1.1 Create new Elixir project

mix new buckyverse --sup
cd buckyverse

1.2 Configure mix.exs

Dependencies:

• yaml_elixir ~> 2.9 - YAML parsing for metadata
• nimble_parsec ~> 1.4 - Parser combinators for LaTeX
• floki ~> 0.36 - HTML parsing
• slugify ~> 1.3 - URL slug generation
• rambo ~> 0.3 - Shell commands (tex4ht fallback)
• briefly ~> 0.5 - Temp files
• file_system ~> 1.0 - File watching
• jason ~> 1.4 - JSON encoding

Mix aliases:

• bucky.build - Build static site
• bucky.watch - Dev server with live reload

1.3 Create directory structure

Create all directories listed in project structure above.

Phase 2: Data Structures

2.1 Book struct (lib/buckyverse/library/book.ex)

Fields:

• slug - URL identifier
• title - Book title
• subtitle - Optional subtitle
• author - Author name
• year - Publication year
• description - Book description
• chapters - List of Chapter structs

2.2 Chapter struct (lib/buckyverse/library/chapter.ex)

Fields:

• slug - URL identifier
• title - Chapter title
• section - Section number (e.g., "100")
• position - Sort order
• pages - List of Page structs

2.3 Page struct (lib/buckyverse/library/page.ex)

Fields:

• slug - URL identifier
• book_slug - Parent book
• chapter_slug - Parent chapter
• title - Page title
• section - Section number (e.g., "100.00")
• position - Sort order
• body - Raw markdown content
• html - Rendered HTML
• toc - Table of contents (list of heading maps)

Phase 3: LaTeX Parser

3.1 Parser (lib/buckyverse/latex/parser.ex)

Use NimbleParsec to build a LaTeX math parser.

Tokens to support:

• Numbers: 123, 3.14, -5
• Variables: single letters a-z, A-Z
• Greek letters: \alpha, \beta, \gamma, etc. (lowercase and uppercase)
• Binary operators: +, -, =, <, >, \cdot, \times, \pm, \leq, \geq, \neq, \approx, \in, \subset, \cup, \cap, \to, \Rightarrow
• Big operators: \sum, \prod, \int, \lim, \sin, \cos, \log, \ln
• Delimiters: (, ), [, ], \{, \}, |, \left, \right, \langle, \rangle

Structures to support:

• Superscript: x^2, x^{2+1}
• Subscript: x_1, x_{i+1}
• Fraction: \frac{num}{denom}
• Square root: \sqrt{x}, \sqrt[n]{x}
• Binomial: \binom{n}{k}
• Text: \text{...}, \mathrm{...}
• Accents: \overline, \hat, \vec, \dot, \tilde
• Matrices: \begin{pmatrix}...\end{pmatrix}, bmatrix, vmatrix
• Braced groups: {...}

Parser output: AST with tagged tuples like:

• {:num, "5"}
• {:var, "x"}
• {:op, "+"}
• {:superscript, base, exponent}
• {:frac, [numerator, denominator]}
• {:sqrt, content} or {:sqrt, [{:index, n}, content]}
• {:matrix, type, rows}

3.2 MathML Renderer (lib/buckyverse/latex/mathml.ex)

Convert AST to MathML string.

MathML elements to generate:

• <math> - Root element with xmlns and optional display="block"
• <mi> - Identifier (variable)
• <mn> - Number
• <mo> - Operator
• <mrow> - Horizontal grouping
• <msup> - Superscript
• <msub> - Subscript
• <msubsup> - Both sub and superscript
• <mfrac> - Fraction
• <msqrt> - Square root
• <mroot> - Nth root
• <mover> - Accent over
• <munder> - Accent under
• <mtable>, <mtr>, <mtd> - Matrix
• <mtext> - Text

Public API:

• render(ast, opts) - Returns MathML string
• Options: display: true/false for block vs inline

3.3 High-level LaTeX module (lib/buckyverse/latex.ex)

Public API:

• to_mathml(latex_string, opts) - Returns {:ok, mathml} or {:error, reason}
• to_mathml!(latex_string, opts) - Returns mathml or error span HTML

Try pure Elixir parser first, fall back to tex4ht if configured.

3.4 tex4ht fallback (lib/buckyverse/latex/tex4ht.ex)

For complex expressions the parser can't handle:

1. Build minimal LaTeX document
2. Run make4ht with mathml option
3. Extract <math> element from output
4. Return MathML string

Requires TeX Live with tex4ht installed on system.

Phase 4: Markdown Renderer

4.1 Renderer (lib/buckyverse/markdown/renderer.ex)

Process content in order:

1. Convert display math - Replace $$...$$ with MathML block div
2. Convert inline math - Replace $...$ with MathML inline
3. Convert markdown - Headers, bold, italic, links, images, code blocks, paragraphs
4. Extract TOC - Find all h2/h3/h4 with IDs, return list of %{level, id, text}

Header conversion:

• Add id attribute based on slugified text
• Support h1-h6

Paragraph handling:

• Split on double newlines
• Wrap non-special content in <p> tags
• Don't wrap divs, headers, pre, figure

Return: {html, toc}

Phase 5: Content Loading

5.1 Library module (lib/buckyverse/library.ex)

Public API:

• list_books() - Returns list of Book structs
• get_book(slug) - Returns {:ok, book} or {:error, :not_found}
• get_page(book_slug, chapter_slug, page_slug) - Returns page or error
• books_path() - Returns path to priv/books

Loading logic:

1. Scan priv/books/ for directories
2. For each book directory, load _book.yaml
3. Scan for chapter directories (matching ^\d+-)
4. For each chapter, load _chapter.yaml if exists
5. Scan for .md files in chapter
6. Parse frontmatter and render content for each page
7. Build nested struct hierarchy

Frontmatter parsing:

• Detect --- delimiters
• Parse YAML between delimiters
• Return {frontmatter_map, body_string}

Slug extraction:

• Remove leading digits and hyphen from directory/file names
• 100-universe → universe
• 01-introduction.md → introduction

Phase 6: HTML Templates

6.1 Layout template (lib/buckyverse/templates/layout.ex)

Full HTML document with:

• DOCTYPE, html, head, body
• Meta charset and viewport
• Title from assigns
• Link to /css/style.css
• Content slot
• Script tag for /js/app.js

6.2 Page template (lib/buckyverse/templates/page.ex)

Three-column layout:

• Sidebar - Book title, chapter/page navigation tree
• Content - Page header (section number, title), rendered HTML, prev/next links
• TOC - "On this page" with heading links

6.3 Components (lib/buckyverse/templates/components.ex)

sidebar(book, nav_tree, current_page):

• Book title linking to book index
• Chapters with nested page lists
• Active state for current page
• Section numbers if present

page_nav(book, prev_page, next_page):

• Previous link if exists
• Next link if exists

toc(items):

• Heading list with anchor links
• Indentation by level

6.4 Home template (lib/buckyverse/templates/home.ex)

• Site title "Buckyverse"
• Tagline
• Grid of book cards with title, author, description

6.5 Book index template

• Book title, subtitle, author
• List of chapters with their pages

Phase 7: Static Site Builder

7.1 Build module (lib/buckyverse/build.ex)

run() function:

1. Delete _site/ if exists
2. Create _site/
3. Build all books (pages as nested directories)
4. Build home page
5. Copy static assets
6. Build search index JSON

For each book:

1. Create _site/{book_slug}/
2. Write index.html (book index)
3. For each page:
   • Create _site/{book_slug}/{chapter_slug}/{page_slug}/
   • Calculate prev/next pages
   • Render page template
   • Wrap in layout
   • Write index.html
4. Copy figures/ directory if exists

Search index:

• Array of objects with: title, section, book, url, text (first 500 chars)
• Write to _site/search-index.json

Phase 8: Development Server

8.1 Watch module (lib/buckyverse/watch.ex)

1. Run initial build
2. Start FileSystem watcher on priv/books/ and priv/static/
3. Start simple HTTP server on port 4000 serving _site/
4. On file change events, rebuild affected content
5. Log rebuild activity

Use :inets httpd or :plug_cowboy for HTTP server.

Phase 9: Styling

9.1 CSS (priv/static/css/style.css)

Layout:

• .reader - Flex container for three columns
• .sidebar - Fixed position, 280px width, scrollable
• .content - Main area with max-width
• .toc - Fixed position on right

Sidebar styling:

• Book title
• Chapter headings (small caps)
• Page links with hover/active states
• Section numbers in monospace

Content styling:

• Prose typography (font sizes, line height, margins)
• Section number display
• Heading styles

Math styling:

• .math-block - Centered, overflow scroll
• math element - Math font family
• .math-error - Red monospace for parse failures

Navigation:

• .page-nav - Flex space-between for prev/next

Responsive:

• Hide TOC below 1200px
• Stack sidebar above content below 768px

Phase 10: Sample Content

10.1 Book metadata (priv/books/synergetics/_book.yaml)

title: "Synergetics"
subtitle: "Explorations in the Geometry of Thinking"
author: "R. Buckminster Fuller"
year: 1975
description: |
  Fuller's comprehensive exploration of geometric thinking
  and the coordinate system of nature.

10.2 Chapter metadata (priv/books/synergetics/100-universe/_chapter.yaml)

title: "Universe"
section: "100"

10.3 Sample page (priv/books/synergetics/100-universe/01-universe.md)

---
title: Universe
section: "100.00"
---

## 100.00 Universe

**100.01** Universe is the aggregate of all humanity's consciously 
apprehended and communicated experience.

**100.02** The fundamental equation relating energy and mass is:

$$E = mc^2$$

where $m$ represents mass and $c$ the speed of light.

**100.03** Consider the quadratic formula:

$$\frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$$

**100.04** A matrix example:

$$\begin{pmatrix} a & b \\ c & d \end{pmatrix}$$

Build & Run Commands

# Install dependencies
mix deps.get

# Build static site
mix bucky.build

# Development server with watch
mix bucky.watch

# Preview built site
cd _site && python3 -m http.server 8000

# Test math parser in IEx
iex -S mix

IEx testing:

alias Buckyverse.LaTeX
alias Buckyverse.LaTeX.{Parser, MathML}

# Parse and render
{:ok, ast} = Parser.parse("E = mc^2")
MathML.render(ast)

# High-level API
LaTeX.to_mathml!("\\frac{a}{b}", display: true)

Deployment

Static output in _site/ can be deployed to:

• Netlify: netlify deploy --prod --dir=_site
• Cloudflare Pages: Connect repo, set build command and output dir
• GitHub Pages: Push _site/ to gh-pages branch
• S3 + CloudFront: Sync to bucket

Future Enhancements

1. Full-text search - Client-side with Lunr.js or server-side with Haystack
2. PDF generation - Use tex4ht or Tectonic for downloadable PDFs
3. Phoenix LiveView - Dynamic version with instant navigation
4. Ash Framework - Structured domain layer for content
5. Cross-references - Parse \ref{} and \label{} for internal linking
6. Bibliography - BibTeX support with citations
7. Figures - Enhanced figure handling with captions and numbering
8. Comments/Annotations - User annotations with persistence

Key Implementation Notes

LaTeX Parser Tips

• Use defcombinatorp for recursive parsers (braced groups, expressions)
• Handle optional whitespace between tokens
• Collect subscript/superscript in post_traverse
• Greek letters must be checked before general variable parser
• Commands with backslash must be checked before single-char operators

MathML Tips

• Always include xmlns="http://www.w3.org/1998/Math/MathML"
• Use display="block" for display math
• Wrap multiple siblings in <mrow>
• Single children don't need <mrow>
• Escape <, >, & in text content

Content Loading Tips

• Sort chapters and pages by directory/file name (numeric prefix)
• Handle missing metadata files gracefully
• Cache rendered content if performance becomes an issue
• Use @external_resource for recompilation on content change (if compiling at build time)

Estimated Timeline

Total: ~16-20 hours

This plan is self-contained and can be executed in a new chat session. Start with Phase 1 and work through sequentially. Each phase builds on the previous ones.