I previously looked at
groff for typesetting and started down a path of the different
options for creating diagrams from text files. While I had thought I
might get along with pic I have instead found ways to
avoid learning any new tools.
pic or dpic
I have looked into pic before but didn't give it proper
consideration, thinking it was relegated to preparing printable
documents like TikZ.
What sparked a new interest was
finding dpic,
which allows for a variety of different output formats most
interesting to me is SVG. So far as I know, most
of dpic is compatible with pic save those
few minor differences highlighted in the documentation like fill
color defaults etc.
I have used a number of different tools for diagrams, workflows, and miscellaneous pictographic needs. A non-exhaustive list includes PlantUML, Mermaid, graphviz, ditaa, pikchr and of hand-written SVGs. Of them all, writing SVGs by hand comes closest to what I am interested in producing. Other tools tend to miss the mark for me because they are too specific to any one application or context. Fighting node weights in graphviz to achieve a "nicer" layout is an exercise in futility, so too is producing something repeatable in ditaa.
As an example of a relatively basic diagram that I created previously in plain old SVG is this diagram used to explain some particulars of configuring HAProxy:
<svg xmlns="http://www.w3.org/2000/svg"
width="360" height="675">
<!-- implied / phantom machines -->
<rect x="30" y="1" width="300" height="200" fill="#fff" stroke="#000" rx="6" ry="6" stroke-dasharray="5 3 9 2"/>
<rect x="30" y="195" width="300" height="50" fill="#fff" stroke="#000" stroke-dasharray="5 3 9 2"/>
<rect x="20" y="5" width="300" height="200" fill="#fff" stroke="#000" rx="6" ry="6" stroke-dasharray="5 3 9 2"/>
<rect x="20" y="199" width="300" height="50" fill="#fff" stroke="#000" stroke-dasharray="5 3 9 2"/>
<!-- haproxy B-->
<rect x="10" y="200" width="300" height="50" fill="#fff" stroke="#000"/>
<text x="120" y="235">HAProxy</text>
<!-- machine B -->
<rect x="10" y="10" width="300" height="200" fill="#ffefcc" stroke="#000" rx="6" ry="6"/>
<text x="20" y="30">Machine B</text>
<text x="70" y="270">10.0.0.2</text>
<!-- replica db -->
<rect x="180" y="100" width="80" height="50" stroke="#000" fill="#fff"/>
<ellipse cx="220" cy="100" rx="40" ry="10" stroke="#000" fill="#fff"/>
<ellipse cx="220" cy="150" rx="40" ry="10" stroke="#000" fill="#fff" stroke-dasharray="5 3 9 2"/>
<text x="220" y="130" text-anchor="middle">replica</text>
<!-- app boxes -->
<rect x="15" y="45" width="120" height="160" fill="white" stroke="#000" rx="6" ry="6"/>
<text x="110" y="65" transform="rotate(90,110,65)">namespace</text>
<rect x="20" y="50" width="70" height="70" fill="#85c585" stroke="green" rx="6" ry="6"/>
<text x="25" y="70">app 3</text>
<rect x="20" y="130" width="70" height="70" fill="#85c585" stroke="green" rx="6" ry="6"/>
<text x="25" y="150">app 4</text>
<!-- VPN -->
<text x="50" y="315">WireGuard</text>
<text x="100" y="335">VPN</text>
<rect x="145" y="250" width="30" height="200" fill="none" stroke="#000" stroke-dasharray="5 3 9 2"/>
<line x1="160" y1="250" x2="160" y2="400" stroke-width="3" stroke="#0A0"/>
<!-- haproxy A-->
<rect x="10" y="590" width="300" height="50" fill="#fff" stroke="#000"/>
<text x="120" y="625">HAProxy</text>
<text x="150" y="660" text-anchor="middle">↑ incoming requests ↑</text>
<!-- machine A -->
<rect x="10" y="400" width="300" height="200" fill="#e0e9ff" stroke="#000" rx="6" ry="6"/>
<text x="20" y="420">Machine A</text>
<text x="70" y="390">10.0.0.1</text>
<!-- db -->
<rect x="180" y="495" width="80" height="50" stroke="#000" fill="#fff"/>
<ellipse cx="220" cy="495" rx="40" ry="10" stroke="#000" fill="#fff"/>
<ellipse cx="220" cy="545" rx="40" ry="10" stroke="#000" fill="#fff" stroke-dasharray="5 3 9 2"/>
<text x="218" y="525" text-anchor="middle">DB</text>
<!-- app boxes -->
<rect x="15" y="435" width="120" height="160" fill="white" stroke="#000" rx="6" ry="6"/>
<text x="110" y="455" transform="rotate(90,110,455)">namespace</text>
<rect x="20" y="440" width="70" height="70" fill="#85c585" stroke="green" rx="6" ry="6"/>
<text x="25" y="460">app 1</text>
<rect x="20" y="520" width="70" height="70" fill="#85c585" stroke="green" rx="6" ry="6"/>
<text x="25" y="550">app 2</text>
<text x="335" y="20" transform="rotate(90,335,20)">additional machines</text>
</svg>
While the result is fine, writing it out was tedious for the number
of duplicated values and changes necessary to shift things
around. SVG provides little in the way of abstraction over constants
etc. instead relying on CSS and JavaScript. I am fond of the style
of diagrams produced by the troff family of tools,
probably because of anything written by R. W. Stevens. To explore
the potential there I was pleased to
find dpic
which brings some troff tooling into the present century by adding
support for SVG as an output target.
The language is pretty sparse, though it does provide a kind of macro
definition facility. While the author of dpic has
produced some pretty great results it seems they rely on M4 of all
things. It has been years since I used M4 so below I try recreating
the diagram using only the built-in macro facility:
.PS
define db { [
UpperE: ellipse width 1 height 0.25
LowerE: ellipse width 1 height 0.25 at (UpperE.center.x, UpperE.center.y+0.65)
line from UpperE.left to LowerE.left
line from UpperE.right to LowerE.right
] }
define stackProxy { [
Larger: box rad 0.025 width 4 height 3 shaded "white"
box width 4 at (Larger.center.x, Larger.b.y+0.25) shaded "white" "HAProxy"
] }
define nsGroup { [
corner = 0.025
half = 0.5
Namespace: box rad corner width 1.5 height 1.5 shaded "white"
"namespace" "" at last box.bottom
App: box rad corner width half height half at ((Namespace.center.x-0.4), (Namespace.center.y+0.3)) "app"
box rad corner width half height half at (App.center.x, App.center.y-0.6) "app"
box rad corner width half height half at (App.center.x+0.6, App.center.y) "app"
] }
Worker1: stackProxy()
Worker2: stackProxy() at (Worker1.center.x-0.1, Worker1.center.y-0.1)
Worker: stackProxy() at (Worker2.center.x-0.1, Worker2.center.y-0.1)
"Machine B" at Worker.l.x+0.6, Worker.b.y+0.6
nsGroup() at (Worker.left.x+0.9, Worker.top.y-1)
db() at (Worker.center.x+1, Worker.center.y) "replica"
move down 3 from (1.8,0)
Main: stackProxy() at Worker.bottom.x, Worker.bottom.y-3
"Machine A" at Main.l.x+0.6, Main.b.y+0.6
line dotted from (Worker.bottom.x+0.2,Worker.bottom.y) to (Main.top.x+0.2,Main.top.y)
line from Worker.bottom to Main.top
line dotted from (Worker.bottom.x-0.2,Worker.bottom.y) to (Main.top.x-0.2,Main.top.y)
"WireGuard VPN" at last line.c.x+1.2, last line.c.y
nsGroup() at (Main.left.x+0.9, Main.top.y-1)
db() at (Main.center.x+1, Main.center.y) "database"
.PE
Things I liked:
pic header and footer!) and doesn't seem
to support stdin directly
Dwight Aplevich reached out to me to tell me dpic does
in fact support reading from stdin. It is helpfully documented on
page two
of the
manual and I can only blame myself for having missed this. The
feedback was helpful and obviously I'll be spending some time
reading the manual more closely. With the knowledge that it is
possible I found the following bit of elisp sufficient to make my
"preview workflow" much nicer than it was previously:
(progn
(shell-command-on-region (region-beginning) (region-end)
"dpic -v" "*SVG*" nil "*ERROR*" t)
(pop-to-buffer "*SVG*")
(image-mode))
Now I get a nice preview of the SVG output of dpic in a
temporary buffer given the selected region (or org-mode source block
etc.). Obviously it pays to read the manual! The rest of this post
is less well motivated now that my main sticking point seems to be
"one more DSL" — making my own isn't exactly going to do me any
favors.
With the above two lists I got to thinking of how I would like to create these sorts of things. I want something moderately well integrated into my workflow, easily changeable, amenable to abstractions I haven't yet thought of. Rolling those ideas around in my head it occurred to me to see whether emacs has an SVG library and it does! I spend most of my day in emacs programming and writing prose, so integration is pretty well assured. As for repetition and abstraction it is all just elisp so it supports nearly anything with varying levels of effort. As a first pass I came up with the following:
(require 'svg)
(defun prelude ()
(fset 'rect (apply-partially #'svg-rectangle *SVG*))
(fset 'line (apply-partially #'svg-line *SVG*))
(fset 'text (apply-partially #'svg-text *SVG*))
(fset 'ellipse (apply-partially #'svg-ellipse *SVG*))
(fset 'polygon (apply-partially #'svg-polygon *SVG*)))
(defun padded-line (x1 y1 x2 y2 padding primary-color secondary-color)
(line (- x1 padding) y1 (- x2 padding) y2 :stroke secondary-color)
(line (+ x1 padding) y1 (+ x2 padding) y2 :stroke secondary-color)
(line x1 y1 x2 y2 :stroke primary-color :stroke-width "3"))
(defun cylinder (x y width height depth stroke fill text)
(polygon (list (cons x (- y (/ depth 2)))
(cons (+ x width) (- y (/ depth 2)))
(cons (+ x width) (+ y height))
(cons x (+ y height)))
:fill fill :stroke stroke)
(ellipse (+ x (/ width 2)) (- y (/ depth 2)) (/ width 2) depth :stroke stroke :fill fill)
(ellipse (+ x (/ width 2)) (+ height y) (/ width 2) depth :stroke stroke :fill fill)
(text text :x (+ x (/ width 2)) :y (+ y (/ height 2)) :text-anchor "middle"))
(defun machine (label db-label fill app-index lhs top width height)
(rect lhs top width height :fill fill :stroke "black" :rx 6 :ry 6)
(text label :x (+ lhs 10) :y (+ top 20))
(rect lhs (+ top (- height 10)) width 50 :fill "white" :stroke "black")
(text "HAProxy" :x (+ lhs (/ width 2)) :y (+ top height 20) :text-anchor "middle")
(rect (+ lhs 5) (+ top 35) 162 175 :fill "white" :stroke "black" :rx 6 :ry 6)
(text "namespace" :x (+ lhs 50) :y (+ top 205))
(cl-loop for (x y) on (list (+ lhs 10) (+ top 40)
(+ lhs 10) (+ top 120)
(+ lhs 90) (+ top 40))
by #'cddr
for i = app-index then (+ 1 i)
do (progn (rect x y 70 70 :fill "#85c585" :stroke "green" :rx "6" :ry "6")
(text (format "app %d" i) :x (+ x 5) :y (+ y 20))))
(cylinder (+ lhs (* width 0.65)) (+ top 50) 80 50 6 "black" "white" db-label))
(let ((*SVG* (svg-create 650 750))
(lhs 10)
(top 10)
(lower 400)
(height 230)
(width 300))
(prelude)
(padded-line (+ lhs (/ width 2)) (+ top height) (+ lhs (/ width 2)) lower 8 "#85c585" "black")
(text "WireGuard VPN" :x (+ lhs (/ width 2) 30) :y (- lower 60))
(dolist (xy '((12 8) (6 4)))
(rect (+ (car xy) lhs) (+ (cadr xy) top) width (+ height 40)
:fill "white" :stroke "black" :rx 6 :ry 6 :stroke-dasharray "5 3 9 2"))
(machine "Machine A" "database" "#e0e9ff" 1 lhs lower width height)
(text "↑ incoming requests ↑" :x (/ width 2) :y (+ lower height 70) :text-anchor "middle")
(machine "Machine B" "replica" "#ffefcc" 3 lhs top width height)
(text "additional machines" :x (+ width 40) :y (* -1 lhs)
:transform (format "rotate(90) translate(%d %d)" (* -1 width) (- (* -1 width) lhs lhs)))
(with-output-to-temp-buffer "*SVG-OUTPUT*"
(with-current-buffer "*SVG-OUTPUT*"
(svg-print *SVG*)
(image-mode))))
I'm an not yet totally satisfied with a number of aspects, like the
repeated use of the svg- prefix. While I appreciate the
brevity of aliasing the function and paritally applying the SVG
context, the use of dynamic scoping is a little zany and the editor
support suffers by losing context for the arguments to the aliased
functions.
That being said, it was easy to get started because I already know
elisp (more or less) and the editor integration is great. I think
what I would like to work out is a reasonably nice approximation to
the way pic addresses relative locations. The label mechanism
coupled with North, East, South, West makes it a breeze to link the
lower-right (South-East) of a given figure to another. I would like
to think of a way to do something similar in elisp. While it
possible to calculate midpoints on the fly like (+ (/ height
2) offset) it is a bit of drudgery that the computer can
probably handle better if only I can tell it how. I actually prefer
the raw SVG style of rounded corners and for stroke decoration. More
than anything I like the brevity of: db() at
(Worker.center.x+1, Worker.center.y) "replica" my current
best attempt of (cylinder (+ lhs (* width 0.65)) (+ top 50) 80
50 6 "black" "white" db-label) isn't horrbile but leaves
something to be desired.
In general I'm looking forward to trying out this new approach. I'm happy to stick with plain SVG rather than a more niche format like pic, despite some of the compelling features. Emacs Lisp is still a fun language to mess around with so I'm looking forward to exploring some features that I haven't yet needed in search of making more diagrams ergonomically.