Markdown tools

So you have folder with markdown files. It can be static website, it can be Obsidian vault, second brain, digital garden, zettelkasten, notes…

And you want to do something with it. There are a lot tools for that:

• “second brain” editors: Obsidian, Foam, Roam Research, etc.
• static website generators: Hugo, Astro, etc.

And everything in between:

• Language servers: VSCode Markdown language server, Marksman
• Graph vizualization: markdown-links, markmap.js, dundalek/markmap
• Transformers:
  • from Obsidian to Hugo: obsidian-export obsidian-to-hugo, obsidian-meets-hugo, obyde
  • to convert code fences to diagrams: pikchr-cmd, rehype-mermaidjs, remark-mermaidjs
  • code formaters: prettier, cbfmt
• Broken link checkers: remark-lint-no-dead-urls, mdv, markdown-link-check, remark-validate-links
• “Treat as database” tools: docsql, obsidian-dataview

One more tool #

We already have plenty. What shall we do? Obviously, create another one.

What I want minimum:

• Given there is a folder with markdown files
• it (tool) reads files and parses markdown
• it resolves wikilinks and relative links
• (optional) it stores data in embedded database, for example SQLite
• it builds graph (as a data structure, not a visualisation)
  • which would allow to check for (local) broken links (also images, header-ids)
  • build visualisation
  • export graph - to work with graph in other tools, like NetworkX etc.
• It dumps files (with resolved links) to another folder
• It dumps grpah structure to JSON file (or any other format)

What I want maximum:

• watch mode
• UI (TUI or GUI) to query data with SQL/Cypher/Datalog…
• plugins, for example:
  • to convert code fences to diagrams (inline or vfile)
  • to generate social images
• LSP
• implement as library so other software could reuse it, for example, Astro, VScode, Obsidian, etc.
• linter and pretty-printer

Preliminary ideas about implementation #

• If we need embeded database, there are:
  • SQLite + Graph-in-RDBMS (there is WASM version)
  • cozo (there is WASM version)
  • kuzu
• fdir - fast directory scanning
• chokidar - file watcher
• maybe we will need something like vfile or browserfs
• there are a lot of markdown parsers. micromark and co has the biggest number of plugins. Plus there is markdown-rs, which would be possible to compile to WASM

Related #

• markdown-to-sqlite
• Using sqlite3 as a notekeeping document graph with automatic reference indexing
• markdowndb

Read more: Documentation generators, BEOE