first draft

Resources/blog/published/mapping-aardwolf.md

@@ -0,0 +1,165 @@
+---
+draft: false
+title: "Mapping Aardwolf with Graphviz and Golang"
+aliases: ["Mapping Aardwolf with Graphviz"]
+series: ["mapping-aardwolf"]
+date: "2023-04-06"
+author: "Nick Dumas"
+cover: ""
+keywords: [""]
+description: "Maxing out your CPU for fun and profit with dense graphs, or how I'm attempting to follow through on my plan to work on projects with more visual outputs"
+showFullContent: false
+tags:
+- graphviz
+- graph
+- aardwolf
+- golang
+---
+
+## Textual Cartography
+Aardwolf has a fairly active developer community, people who write and maintain plugins and try to map the game world and its contents.
+
+I saw one user, Danj, talking about their work on mapping software and my interest was piqued.
+
+The MUSHclient [bundle](https://github.com/fiendish/aardwolfclientpackage/wiki/) provided by Fiendish has a mapper that tracks your movement through ==rooms== and ==exits==. This data is leveraged by a bunch of plugins in a variety of ways, none of which are super relevant to this post.
+
+In practice, I know that I can't possibly compete with existing solutions like the [Gaardian Roominator](http://rooms.gaardian.com/index.php) and the beta SVG version that I don't have a link to at the moment. That doesn't stop me from wanting to gets my hands on the data and see if I can do anything cool with it.
+## The Data
+The mapper's map data is stored in an sqlite database, and the schema is pretty straightforward. There's a few tables we care about: [[#Areas]], [[#Rooms]], and [[#Exits]].
+
+These tables look like they have a lot of columns, but most of them end up being irrelevant in the context of trying to create a graph representing the rooms and exits connecting them.
+
+The `exits` table is just a join table on `rooms`, so in theory it should be pretty trivial to assemble a list of vertices ( rooms ) and edges ( exits ) and pump them into graphviz, right?
+
+### Areas
+```sql
+sqlite> .schema areas
+CREATE TABLE areas(
+uid TEXT NOT NULL,
+name TEXT,
+texture TEXT,
+color TEXT,
+flags TEXT NOT NULL DEFAULT '',
+`id` integer,
+`created_at` datetime,
+`updated_at` datetime,
+`deleted_at` datetime,
+PRIMARY KEY(uid));
+CREATE INDEX `idx_areas_deleted_at` ON `areas`(`deleted_at`);
+```
+###  Rooms
+```sql
+sqlite> .schema rooms
+CREATE TABLE rooms(
+uid TEXT NOT NULL,
+name TEXT,
+area TEXT,
+building TEXT,
+terrain TEXT,
+info TEXT,
+notes TEXT,
+x INTEGER,
+y INTEGER,
+z INTEGER,
+norecall INTEGER,
+noportal INTEGER,
+ignore_exits_mismatch INTEGER NOT NULL DEFAULT 0,
+`id` integer,
+`created_at` datetime,
+`updated_at` datetime,
+`deleted_at` datetime,
+`flags` text,
+PRIMARY KEY(uid));
+CREATE INDEX rooms_area_index ON rooms (area);
+CREATE INDEX `idx_rooms_deleted_at` ON `rooms`(`deleted_at`);
+```
+
+It wasn't until writing this and cleaning up that `CREATE TABLE` statement to be readable did I notice that rooms have integer IDs. That may be useful for solving the problems I'll describe shortly.
+###  Exits
+```sql
+sqlite> .schema exits
+CREATE TABLE exits(
+dir TEXT NOT NULL,
+fromuid TEXT NOT NULL,
+touid TEXT NOT NULL,
+level STRING NOT NULL DEFAULT '0',
+PRIMARY KEY(fromuid, dir));
+CREATE INDEX exits_touid_index ON exits (touid);
+```
+## Almost Right
+Getting the edges and vertices into graphviz ended up being pretty trivial. The part that took me the longest was learning how to do database stuff in Go. So far I'd managed to interact with flat files and HTTP requests for getting my data, but I knew that wouldn't last forever.
+### A relational tangent
+In brief, the Go database workflow has some steps in common:
+1) import `database/sql`
+2) import your database driver
+3) open the database or establish a connection to the server
+4) Make a query
+5) Scan() into a value
+6) use the value
+
+There's some variance with points 5 and 6 on whether you want exactly one or some other number of results ( `Query` vs `QueryRow`) .
+
+To demonstrate, here's a pared down sample of what I'm using in my `aardmapper`.
+
+```go {title="main.go"}
+import (
+  "database/sql"
+  _ "github.com/mattn/go-sqlite3"
+)
+
+type Area struct {
+  Uid, Name, Flags, Color, Texture sql.NullString
+}
+
+func main() {
+  db, _ := sql.Open("sqlite3", fn)
+  // error handling is elided for brevity. do not ignore errors.
+}
+  a = Area{}
+  if err := row.Scan(&a.Uid, &a.Name, &a.Flags, &a.Color, &a.Texture); err != nil {
+    if err == sql.ErrNoRows {
+      fmt.Fatalf("no area found: %w", err)
+    }
+  }
+  // do stuff with your queried Area
+```
+##  The graph must grow
+Once I was able to query rooms and exits from the database, I was on the fast track. The graphviz API is relatively straightforward when you're using Go:
+```go {title="mapper.go"}
+gv := graphviz.New()
+g := gv.Graph()
+for _, room := range rooms { // creation of rooms elided
+    origin, _ := g.CreateNode("RoomID_AAAAA")
+    dest, _ := g.CreateNode("RoomID_AAAAB")
+    edge, _ := g.CreateEdge("connecting AAAAA to AAAAB", origin, dest)
+}
+// Once again, error handling has been elided for brevity. **Do not ignore errors**.
+```
+
+This ended up working great. The rooms and exits matched up to vertices and edges the way I expected.
+
+The only problem was that rendering the entire thing on my DigitalOcean droplet will apparently take more than 24 hours. I had to terminate the process at around 16 hours because I got impatient.
+## The lay of the land
+This first, naive implementation mostly does the trick. It works really well for smaller quantities of rooms. Below you can see a PNG and SVG rendering of 250 rooms, and the code used to generate it.
+
+```go
+  if err := gv.RenderFilename(g, graphviz.SVG, "/var/www/renders.ndumas.com/aardmaps/name.ext"); err != nil {
+    log.Fatal(err)
+  }
+```
+
+{{< figure src="[[Resources/attachments/250-rooms.svg]]" title="250 Rooms (SVG)" alt="a disorderly grid of squares representing rooms connected to each other in a video game" caption="SVG scales a lot better" >}}
+
+
+{{< figure src="[[Resources/attachments/250-rooms.png]]" title="250 Rooms (PNG)" alt="a disorderly grid of squares representing rooms connected to each other in a video game" caption="Raster images can be simpler and more performant to render" >}}
+
+
+## What's next?
+The current iteration of rendering is really crude:
+- The rooms are displayed using their numeric IDs, not human friendly names.
+- Rooms are grouped by area, creating subgraphs to describe them will help interpreting the map and probably help rendering.
+- The current iteration is very slow
+
+I've also been contemplating the idea of rendering each area one at a time, and then manipulating the resulting SVG to display connections that cross between areas. This would almost certainly be infinitely faster than trying to render 30,00 vertices and 80,000 edges simultaneously.
+
+All my code can be found [here](https://code.ndumas.com/ndumas/aardmapper). It's still early in prototyping so I don't have any stable builds or tags yet.

go.mod

@@ -0,0 +1,10 @@
+module testo
+
+go 1.19
+
+require go.uber.org/zap v1.24.0
+
+require (
+	go.uber.org/atomic v1.7.0 // indirect
+	go.uber.org/multierr v1.6.0 // indirect
+)

go.sum

@@ -0,0 +1,18 @@
+github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
+go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
+go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
+go.uber.org/goleak v1.1.11 h1:wy28qYRKZgnJTxGxvye5/wgWr1EKjmUDGYox5mGlRlI=
+go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
+go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
+go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60=
+go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

main.go

@@ -0,0 +1,107 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"io/fs"
+	"os"
+	// "path/filepath"
+	"strings"
+
+	"go.uber.org/zap"
+)
+
+func NewAttachmentMover() *AttachmentMover {
+	var am AttachmentMover
+	l, _ := zap.NewProduction()
+	am.L = l
+	am.Attachments = make(map[string]bool)
+	am.Posts = make([]string, 0)
+
+	return &am
+}
+
+type AttachmentMover struct {
+	Source, Target string
+	Attachments    map[string]bool
+	Posts          []string
+	L              *zap.Logger
+}
+
+func (am *AttachmentMover) Walk() error {
+	return nil
+}
+
+func (am *AttachmentMover) walk(path string, d fs.DirEntry, err error) error {
+	if err != nil {
+		return err
+	}
+
+	if d.IsDir() {
+		return nil
+	}
+	walkLogger := am.L.Named("Walk()")
+
+	walkLogger.Info("scanning for relevance", zap.String("path", path))
+	if strings.HasSuffix(path, "index.md") {
+		walkLogger.Info("found index.md, adding to index", zap.String("path", path))
+		am.Posts = append(am.Posts, path)
+	}
+
+	if !strings.HasSuffix(path, ".md") && strings.Contains(path, "attachments") {
+		walkLogger.Info("found  attachment file, adding to index", zap.String("path", path))
+		am.Attachments[path] = true
+	}
+	return nil
+}
+
+func (am *AttachmentMover) Move() error {
+	moveLogger := am.L.Named("Move()")
+	moveLogger.Info("scanning posts", zap.Strings("posts", am.Posts))
+	for _, post := range am.Posts {
+		// log.Printf("scanning %q for attachment links", post)
+		linkedAttachments, err := extractAttachments(post)
+		if err != nil {
+			return fmt.Errorf("could not extract attachment links from %q: %w", post, err)
+		}
+		for _, attachment := range linkedAttachments {
+			moveAttachment(post, attachment)
+		}
+	}
+
+	return nil
+}
+
+func moveAttachment(post, attachment string) error {
+
+	return nil
+}
+
+func extractAttachments(fn string) ([]string, error) {
+	attachments := make([]string, 0)
+
+	return attachments, nil
+}
+
+func main() {
+	am := NewAttachmentMover()
+
+	flag.StringVar(&am.Source, "source", "", "source directory containing your vault")
+	flag.StringVar(&am.Target, "target", "", "target directory containing your hugo site")
+
+	flag.Parse()
+
+	if am.Source == "" || am.Target == "" {
+		am.L.Fatal("flags not provided")
+	}
+
+	err := am.Walk()
+	if err != nil {
+		// log.Fatalf("error walking blog dir to gather file names: %s\n", err)
+	}
+
+	err = am.Move()
+	if err != nil {
+		// log.Fatalf("error walking blog dir to gather file names: %s\n", err)
+	}
+}