[GH-ISSUE #188] Replace graphviz2drawio with native mxGraph emitter #113

New issue

Closed

opened 2026-05-06 12:37:36 +02:00 by BreizhHardware · 1 comment

BreizhHardware commented

2026-05-06 12:37:36 +02:00

Owner

Originally created by @patrickchugh on GitHub (Apr 14, 2026).
Original GitHub issue: https://github.com/patrickchugh/terravision/issues/188

Summary

Replace the current graphviz2drawio library with a native mxGraph XML emitter that generates drawio files directly from tfdata["graphdict"] and the existing node/cluster model in modules/drawing.py. As part of the rewrite, wire up drawio's native provider shape libraries (AWS / Azure / GCP) instead of embedding base64 PNG icons.

Motivation

Dependency pain

graphviz2drawio depends on pygraphviz, which is a C-extension Python binding to Graphviz. This causes real pain on:

Apple Silicon (arm64): users must export CFLAGS/LDFLAGS pointing at Homebrew's Graphviz headers before pip install succeeds.
Windows: similar story — needs MSVC + Graphviz dev headers.
Docker/CI: slower builds, more moving parts.

We document a workaround in the README but it's a recurring friction point for users installing terravision[drawio].

Output quality issues

On top of the install pain, the library has several quality issues in its output that we can't easily fix without controlling the emitter:

Footer lands at the top of the canvas because the library ignores our gvpr-placed fixed positions (fixed via XML post-processor in v0.34.x).
Node labels appear above icons instead of below (fixed via XML post-processor).
Cluster corner icons (AWS Cloud, VPC, subnet) are silently dropped — HTML-table <IMG> cluster labels we embed for the PNG path don't survive the conversion. No clean fix without a rewrite.
Custom styles we'd like control over (edge label positions, cluster header heights, bidirectional arrows) are all at the mercy of the library's SVG→mxGraph mapping.

Use native drawio shape libraries (rolled in from #182)

Today we emit every node as shape=image;image=data:image/png,<base64> — the icon is embedded inline. This:

Bloats drawio files by ~20× (600KB vs ~30KB for the same diagram).
Doesn't integrate with drawio's shape picker sidebar.
Doesn't let users right-click → replace with a sibling shape from the same official library.
Looks foreign next to user-added shapes.

drawio ships official libraries for AWS, Azure, and GCP. We should emit shape=mxgraph.aws4.lambda, shape=mxgraph.azure.kubernetes_service, shape=mxgraph.gcp.compute_engine, etc. We can't do this through graphviz2drawio because it hard-codes shape=image, so the work is gated on the rewrite.

Proposed approach

Emit mxGraph XML ourselves using dot's -Txdot output for layout coordinates. The dot binary is already a required system dependency (needed for the PNG path), so we don't gain a dependency — we lose one (pygraphviz / graphviz2drawio).

Pipeline

graphdict + meta_data
  → modules/drawing.py (existing pipeline generates DOT)
  → dot -Txdot postdot.dot  (already a system dep)
  → parse xdot positions (regex; well-documented format)
  → emit mxGraph XML directly with full control of styles/icons/labels
  → .drawio file

What we keep

All existing DOT generation (generate_dot, cluster building, handle_group, handle_nodes).
All gvpr post-processing for positions.
_postprocess_drawio_xml goes away — we produce correct XML from the start.

What we write (estimate ~400–600 lines)

xdot parser (~50 lines): read pos="x,y", bb="x,y,x,y", width, height, lp (label position) from each node/cluster.
mxGraph emitter (~200 lines): <mxGraphModel> / <mxCell> XML with correct hierarchy (parent= from the cluster tree), correct styles per resource type, edges with source/target mxCell IDs.
Provider shape-library mapping (~150 lines): lookup tables mapping our internal resource types to drawio shape names for AWS, Azure, GCP. Reference: ARI's Azure diagram module ships a usable mapping we can port. Fallback to embedded PNG when no shape-library entry exists.
Style library (~100 lines): a small dict mapping our Python Cluster / Node subclasses to mxGraph style strings — we already know every icon, color, and shape per resource.
Tests: verify emitted XML parses, contains expected cells, and passes drawio schema check (simple regex/parse tests).

Benefits

Drop graphviz2drawio and pygraphviz dependencies entirely → no more compile flags on arm64/Windows.
Corner icons work out of the box (we inject them directly at cluster top-left).
Footer positioning correct by construction (no post-fix needed).
Label position / spacing correct from the start.
Native drawio provider shapes — files are ~20× smaller, icons come from drawio's built-in AWS/Azure/GCP libraries, users can replace/augment shapes from the drawio shape picker.
Round-trip lossless — what we emit is exactly what drawio renders.
Faster drawio export (skips re-layout).

Costs

One-time ~400–600 LOC engineering effort.
Maintenance of the provider shape-library mapping (new AWS services → new mappings).
New test coverage: emitted XML must load in drawio, contain all expected nodes/edges/clusters.
Some duplication of icon-resolution / style logic already in drawing.py — worth extracting shared helpers.

Scope boundaries

Does not replace PNG rendering — that path still uses graphviz's dot via the Python Diagram wrapper.
Does not add drawio-native layout algorithms — we continue to use graphviz's layout, just parse and emit it ourselves.
Does not change the CLI surface — terravision draw --format drawio still works identically.

Closes #182 — use the native drawio Azure icons (and by extension AWS / GCP).

References

Current drawio export: modules/drawing.py render_diagram() around the if format == "drawio": block.
Current post-processor workaround: _postprocess_drawio_xml().
xdot format spec: https://graphviz.org/docs/outputs/canon/#xdot
mxGraph XML schema: https://www.drawio.com/doc/faq/mxgraphmodel-xml-model-format
Azure shape mapping reference: https://github.com/microsoft/ARI/blob/main/Modules/Public/PublicFunctions/Diagram/

Originally created by @patrickchugh on GitHub (Apr 14, 2026). Original GitHub issue: https://github.com/patrickchugh/terravision/issues/188 ## Summary Replace the current `graphviz2drawio` library with a native mxGraph XML emitter that generates drawio files directly from `tfdata["graphdict"]` and the existing node/cluster model in `modules/drawing.py`. As part of the rewrite, wire up drawio's **native provider shape libraries** (AWS / Azure / GCP) instead of embedding base64 PNG icons. ## Motivation ### Dependency pain `graphviz2drawio` depends on `pygraphviz`, which is a C-extension Python binding to Graphviz. This causes real pain on: - **Apple Silicon (arm64)**: users must export `CFLAGS`/`LDFLAGS` pointing at Homebrew's Graphviz headers before `pip install` succeeds. - **Windows**: similar story — needs MSVC + Graphviz dev headers. - **Docker/CI**: slower builds, more moving parts. We document a workaround in the README but it's a recurring friction point for users installing `terravision[drawio]`. ### Output quality issues On top of the install pain, the library has several quality issues in its output that we can't easily fix without controlling the emitter: - **Footer lands at the top** of the canvas because the library ignores our gvpr-placed fixed positions (fixed via XML post-processor in v0.34.x). - **Node labels appear above icons** instead of below (fixed via XML post-processor). - **Cluster corner icons (AWS Cloud, VPC, subnet) are silently dropped** — HTML-table `<IMG>` cluster labels we embed for the PNG path don't survive the conversion. No clean fix without a rewrite. - **Custom styles we'd like control over** (edge label positions, cluster header heights, bidirectional arrows) are all at the mercy of the library's SVG→mxGraph mapping. ### Use native drawio shape libraries (rolled in from #182) Today we emit every node as `shape=image;image=data:image/png,<base64>` — the icon is embedded inline. This: - Bloats drawio files by ~20× (600KB vs ~30KB for the same diagram). - Doesn't integrate with drawio's shape picker sidebar. - Doesn't let users right-click → replace with a sibling shape from the same official library. - Looks foreign next to user-added shapes. drawio ships official libraries for AWS, Azure, and GCP. We should emit `shape=mxgraph.aws4.lambda`, `shape=mxgraph.azure.kubernetes_service`, `shape=mxgraph.gcp.compute_engine`, etc. We can't do this through `graphviz2drawio` because it hard-codes `shape=image`, so the work is gated on the rewrite. ## Proposed approach Emit mxGraph XML ourselves using `dot`'s `-Txdot` output for layout coordinates. The `dot` binary is already a required system dependency (needed for the PNG path), so we don't gain a dependency — we lose one (`pygraphviz` / `graphviz2drawio`). ### Pipeline ``` graphdict + meta_data → modules/drawing.py (existing pipeline generates DOT) → dot -Txdot postdot.dot (already a system dep) → parse xdot positions (regex; well-documented format) → emit mxGraph XML directly with full control of styles/icons/labels → .drawio file ``` ### What we keep - All existing DOT generation (`generate_dot`, cluster building, `handle_group`, `handle_nodes`). - All gvpr post-processing for positions. - `_postprocess_drawio_xml` goes away — we produce correct XML from the start. ### What we write (estimate ~400–600 lines) 1. **xdot parser** (~50 lines): read `pos="x,y"`, `bb="x,y,x,y"`, `width`, `height`, `lp` (label position) from each node/cluster. 2. **mxGraph emitter** (~200 lines): `<mxGraphModel>` / `<mxCell>` XML with correct hierarchy (`parent=` from the cluster tree), correct styles per resource type, edges with source/target mxCell IDs. 3. **Provider shape-library mapping** (~150 lines): lookup tables mapping our internal resource types to drawio shape names for AWS, Azure, GCP. Reference: [ARI's Azure diagram module](https://github.com/microsoft/ARI/blob/main/Modules/Public/PublicFunctions/Diagram/Start-ARIDiagramSubscription.ps1) ships a usable mapping we can port. Fallback to embedded PNG when no shape-library entry exists. 4. **Style library** (~100 lines): a small dict mapping our Python `Cluster` / `Node` subclasses to mxGraph style strings — we already know every icon, color, and shape per resource. 5. **Tests**: verify emitted XML parses, contains expected cells, and passes drawio schema check (simple regex/parse tests). ## Benefits - Drop `graphviz2drawio` and `pygraphviz` dependencies entirely → no more compile flags on arm64/Windows. - Corner icons work out of the box (we inject them directly at cluster top-left). - Footer positioning correct by construction (no post-fix needed). - Label position / spacing correct from the start. - **Native drawio provider shapes** — files are ~20× smaller, icons come from drawio's built-in AWS/Azure/GCP libraries, users can replace/augment shapes from the drawio shape picker. - Round-trip lossless — what we emit is exactly what drawio renders. - Faster drawio export (skips re-layout). ## Costs - One-time ~400–600 LOC engineering effort. - Maintenance of the provider shape-library mapping (new AWS services → new mappings). - New test coverage: emitted XML must load in drawio, contain all expected nodes/edges/clusters. - Some duplication of icon-resolution / style logic already in `drawing.py` — worth extracting shared helpers. ## Scope boundaries - Does **not** replace PNG rendering — that path still uses graphviz's `dot` via the Python `Diagram` wrapper. - Does **not** add drawio-native layout algorithms — we continue to use graphviz's layout, just parse and emit it ourselves. - Does **not** change the CLI surface — `terravision draw --format drawio` still works identically. ## Related issues - Closes #182 — use the native drawio Azure icons (and by extension AWS / GCP). ## References - Current drawio export: [`modules/drawing.py`](modules/drawing.py) `render_diagram()` around the `if format == "drawio":` block. - Current post-processor workaround: `_postprocess_drawio_xml()`. - xdot format spec: https://graphviz.org/docs/outputs/canon/#xdot - mxGraph XML schema: https://www.drawio.com/doc/faq/mxgraphmodel-xml-model-format - Azure shape mapping reference: https://github.com/microsoft/ARI/blob/main/Modules/Public/PublicFunctions/Diagram/

BreizhHardware

2026-05-06 12:37:36 +02:00

closed this issue
added the
enhancement
label

BreizhHardware commented

2026-05-06 12:37:37 +02:00

Author

Owner

@patrickchugh commented on GitHub (Apr 16, 2026):

Also auto open draw.io website with generated file from terravision loaded if terravision is run with --show flag

@patrickchugh commented on GitHub (Apr 16, 2026): Also auto open draw.io website with generated file from terravision loaded if terravision is run with --show flag