Well, I've got a less worse name for my site, that is now "Concept Coati". I really liked drawing that coati on the logo.
Well, I've got a less worse name for my site, that is now "Concept Coati". I really liked drawing that coati on the logo.
The Designer is a database designing tool that is currently under development.
It's intended to integrate well with git and a git-based workflow, while also being able to replace the database access applications during development, allowing for the management of the data and schema migration right from the same place.
This is the third part of a multi-page article on an experiment to decide if I can migrate the designer front-end into rust and wasm.
On the first part, there is the motivation and also the creation of a hello world project. Also, the code we will migrate is presented there; it is a module that creates tabbed panels and lets them be assigned into containers.
On the second part there is a first attempt on migration and discussion of the problems it faces.
Now, we get the final form of that code.
The final code is more complex than the original. And granted, it was much harder to write.
That's a normal experience on porting code from high-level languages into Rust. While JavaScript is very easy and fast to write, but extremely difficult to change correctly, Rust is very hard to write, but easy to keep correct even after radical changes.
And this experiment was no exception. It gets even harder to write because of the heavy use of an FFI, but it was changed many times during its creation, and maintaining correctness wasn't a problem at all.
Overall, I am satisfied with the result.
Anyway, let's get to the code.
Cargo.toml
We keep the same dependencies from part 2:
[package]
name = "designer"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[build]
target = "wasm32-unknown-unknown"
[features]
default = ["console_error_panic_hook"]
[dependencies]
wasm-bindgen = "0.2.79"
js-sys = "0.3.56"
web-sys = {version = "0.3.56", features=[
"Element", "HtmlElement", "Document", "DomStringMap",
"HtmlCollection", "DomTokenList", "EventListener",
"MouseEvent", "EventTarget", "console", "HtmlSpanElement",
]}
# Remove before deployment:
console_error_panic_hook = { version = "0.1.6", optional = true }
# Small but slow memory allocator:
wee_alloc = { version = "0.4.5", optional = true }
[dev-dependencies]
wasm-bindgen-test = "0.3.13"
[profile.release]
# Tell `rustc` to optimize for small code size.
opt-level = "s"
src/lib.rs
And remove the "hello world" functionality:
mod utils;
use wasm_bindgen::prelude::*;
// When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
// allocator.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;
pub mod tabs;
src/tabs.rs
Now, the kernel of the experiment.
We will need stuff:
use std;
use std::rc::Rc;
use std::cell::{RefCell, RefMut};
use std::collections::{HashMap};
use wasm_bindgen::{JsValue, JsCast};
use web_sys::{HtmlElement, Event};
use web_sys::console;
use js_sys::{Array, Function};
use wasm_bindgen::prelude::*;
And will also need our custom JavaScript implementations:
#[wasm_bindgen]
extern {
#[wasm_bindgen(catch)]
fn create_html(e: &str) -> Result<HtmlElement, JsValue>;
#[wasm_bindgen]
fn html_by_id(i: &str) -> HtmlElement;
#[wasm_bindgen]
fn has_class(e: &JsValue, class: &str) -> bool;
#[wasm_bindgen(catch)]
fn set_class(e: &JsValue, class: &str) -> Result<(), JsValue>;
#[wasm_bindgen(catch)]
fn clear_class(e: &JsValue, class: &str) -> Result<(), JsValue>;
#[wasm_bindgen(catch)]
fn set_data(e: &JsValue, key: &str, value: &str) -> Result<(), JsValue>;
#[wasm_bindgen]
fn get_data(e: &JsValue, key: &str) -> String;
#[wasm_bindgen(catch)]
fn set_onclick(e: &JsValue, on_click: &Function) -> Result<(), JsValue>;
}
We need that global tab registry:
thread_local! (
static TAB_REGISTRY: Rc<RefCell<HashMap<String, Rc<RefCell<Tab>>>>>
= Rc::new(RefCell::new(HashMap::new()));
);
/// Uses a tab from the registry
fn with_tab_mut<A, F>(tab_name: &str, f: F) -> Result<A, JsValue>
where F: FnOnce(RefMut<Tab>) -> Result<A, JsValue> {
TAB_REGISTRY.with(|all_tabs_ro| {
let mut all_tabs: RefMut<_> = all_tabs_ro.try_borrow_mut()
.map_err(|_| concat!(
"Value is in use ",
"- this is a bug, not a normal error condition"))?;
// Get the tab object:
let tab_ro = all_tabs.get_mut(&tab_name.to_string())
.ok_or(format!("Tab named {} not found", tab_name))?;
let tab = tab_ro.try_borrow_mut()
.map_err(|_| concat!(
"Value is in use ",
"- this is a bug, not a normal error condition"))?;
f(tab)
})
}
We also nee that static event handler:
fn select_tab_handler_fn(e: Event) -> Result<(), JsValue> {
let targ_ev = e.target().ok_or("Tab selection event without a target")?;
let targ = targ_ev.dyn_into::<HtmlElement>()?;
let tab_name = targ.dataset().get("tabname")
.ok_or("Tab element does not have a data-tabname property")?;
with_tab_mut(&tab_name, |mut tab| {
tab.select()?;
tab.last_focus = None;
Ok(())
})
}
thread_local! (
/// Event handler for user initiated tab selection
static SELECT_TAB_HANDLER: Closure<dyn Fn(Event)> = Closure::wrap(
Box::new(move |e: Event| {
match select_tab_handler_fn(e) {
Ok(_) => (),
Err(e) => console::error(&Array::of1(&e)),
}
}) as Box<dyn Fn(Event)>);
);
Also, let's make an empty struct for encapsulating the tab manager functions inside a namespace. It is slightly harder to create real namespaces on the JavaScript interface, and this won't matter at all after the entire front-end is migrated to Rust.
This is not very different from what a global tab manager would look like in JavaScript. There isn't much added complexity on this part of the final code.
/// The class of the selected tabs
const DISP: &str = "display";
/**
Global API for managing tabs
*/
#[wasm_bindgen]
struct Tabs {}
#[wasm_bindgen]
impl Tabs {
/**
Creates a new tab, without assigning it.
*/
pub fn add(tab_name: &str, panel: HtmlElement) -> Result<(), JsValue> {
TAB_REGISTRY.with(|all_tabs_ro| {
let mut all_tabs: RefMut<_> = all_tabs_ro.try_borrow_mut()
.map_err(|_| concat!(
"Value is in use ",
"- this is a bug, not a normal error condition"))?;
let t = Tab::new(tab_name, panel)?;
all_tabs.insert(tab_name.to_string(), Rc::new(RefCell::new(t)));
Ok(())
})
}
/**
Assigns a tab into a container pannel.
The container must have the property "data-tabsid"
with the id of its tab panel.
*/
pub fn assign(tab_name: &str, container: HtmlElement) -> Result<(), JsValue> {
// Notice that the entire implementation moved here
with_tab_mut(tab_name, |tab| {
// Inserts the element
let tabsid = get_data(&container, "tabsid");
let tabs = html_by_id(&tabsid);
container.append_with_node_1(&tab.panel)?;
tabs.append_with_node_1(&tab.tab)?;
// Selects a tab if none is selected
// This is much worse than the javascript original.
let selected = Array::from(&tabs.children()).iter().filter(
|e| {
match e.dyn_ref::<HtmlElement>() {
None => false,
Some(he) => he.class_list().contains(DISP),
}
}).count();
if selected == 0 {
tab.tab.class_list().add_1(DISP)?;
tab.panel.class_list().add_1(DISP)?;
}
// Selection handling
SELECT_TAB_HANDLER.with(
|h| set_onclick(&tab.tab, h.as_ref().unchecked_ref())
)?;
Ok(())
})
}
/// Selects a tab on its container
pub fn select(tab_name: &str) -> Result<(), JsValue> {
with_tab_mut(tab_name, |mut tab| {
tab.select()?;
Ok(())
})
}
/// Selects the previously selected tab from the container
pub fn unselect(tab_name: &str) -> Result<(), JsValue> {
with_tab_mut(tab_name, |mut tab| {
tab.unselect()?;
Ok(())
})
}
}
And finally, we implement what is left of the Tab class, making sure never to call the tab manager or use the registry from here. Again, there isn't much added complexity here either:
/**
A panel with a companion tab that you can assign
into tabbed containers, identified by having the
class="tabspanel" and data-tabsid="centerTabs"
properties.
*/
struct Tab {
tab: HtmlElement,
panel: HtmlElement,
last_focus: Option<String>,
}
impl Tab{
pub fn new(tab_name: &str, panel: HtmlElement) -> Result<Tab, JsValue> {
let tab = create_html("span")?;
tab.set_text_content(Some(tab_name));
set_data(&tab, "tabname", tab_name)?;
panel.dataset().set("tabname", tab_name)?;
Ok(Tab{
tab: tab,
panel: panel,
last_focus: None
})
}
/// Selects this tab as the visible one on the container
pub fn select(&mut self) -> Result<(), JsValue> {
let tabparent = self.tab.parent_element()
.ok_or("Tab element does not have a parent")?;
for e in Array::from(&tabparent.children()).iter() {
clear_class(&e, DISP)?;
}
let panelparent = self.panel.parent_element()
.ok_or("Tab panel element does not have a parent")?;
for e in Array::from(&panelparent.children()).iter() {
clear_class(&e, DISP)?;
}
set_class(&self.tab, DISP)?;
set_class(&self.panel, DISP)?;
self.last_focus = None;
Ok(())
}
/// Goes back one step on the focus stack
pub fn unselect(&mut self) -> Result<(), JsValue> {
match &self.last_focus {
None => (),
Some(last) => {
let tabparent = self.tab.parent_element()
.ok_or("Tab element does not have a parent")?;
for e in Array::from(&tabparent.children()).iter() {
match e.dyn_ref::<HtmlElement>() {
None => (),
Some(he) =>
if he.dataset().get("tabname") == Some(last.to_string()) {
he.class_list().add_1(DISP)?;
} else {
he.class_list().remove_1(DISP)?;
}
}
}
let panelparent = self.panel.parent_element()
.ok_or("Tab panel element does not have a parent")?;
for e in Array::from(&panelparent.children()).iter() {
match e.dyn_ref::<HtmlElement>() {
None => (),
Some(he) =>
if he.dataset().get("tabname") == Some(last.to_string()) {
he.class_list().add_1(DISP)?;
} else {
he.class_list().remove_1(DISP)?;
}
}
}
}
}
self.last_focus = None;
Ok(())
}
}
After compiling:
$ wasm-pack build --target web
We can link the objects on the main project:
$ ln -s ../designer/pkg/designer.js
$ ln -s ../designer/pkg/designer_bg.wasm
And serve our front-end locally:
$ python3 -m http.server
On the original JavaScript module, we need some alterations because of the new tabs API. In particular, we need to import and initialize the wasm:
import designer_init, {Tabs} from "./designer.js";
// lots of local code...
window.onload = async () => {
let designer = await designer_init();
// ... the rest of the frontend goes here
}
And for each tab created this way:
toolsTab = new Tab('Tools', toolsElement);
toolsTab.assignContainer(document.querySelector("#right.tabspanel"));
We must now do it this way:
Tabs.add('Tools', toolsElement);
Tabs.assign('Tools', document.querySelector("#right.tabspanel"));
Similarly, for tabs selected this way:
detailsTab.select();
We must do it this way:
Tabs.select('Details');
Internationalization will become an issue some day, but it can be corrected on the global tab manager without changing the API.
And issues like those can be fixed at all, now that changing the code it easy.
The interface change did introduce bugs on the JavaScript front-end. I am aware of at least one functionality that should be broken, but wasn't detected by my tests. But the very good news is that I don't care about it anymore!
The Designer is a database designing tool that is currently under development.
It's intended to integrate well with git and a git-based workflow, while also being able to replace the database access applications during development, allowing for the management of the data and schema migration right from the same place.
This is the second part of a multi-page article on an experiment to decide if I can migrate the designer front-end into rust and wasm.
On the first part, there is the motivation and also the creation of a hello world project. Also, the code I will migrate is presented there; it is a module that creates tabbed panels and lets them be assigned into containers.
On the third part there are the conclusion and the final code.
Now, let's migrate that module.
Let's create the src/tabs.rs
file with the tabs interface only (for
now, I will use String
as the type of everything that is a DOM
element):
struct Tab {
tab: String, // This will be a DOM element
panel: String, // Also a DOM element
tab_name: String,
last_focus: Option<String> // Focus history
}
impl Tab{
// Those things will return error as soon as
// I get what the error should be.
// For now, let's pretend they can't fail.
fn new(tab_name: String, panel: String) -> Tab {
Tab{
tab: "".to_string(), panel: "".to_string(),
tab_name: tab_name, last_focus: None
}
}
/// Assigns the tab to that panel
fn assign_container(&self, container: String) {
}
/// Selects this tab as the visible one on the container
fn select(&self) {
}
/// Goes back one step on the selection stack
fn unselect(&self) {
}
}
The HtmlElement
interface must be imported from JavaScript, just as all
the functions and objects used within the code, like the global
document
. Those can be imported manually, as needed, but there
exist two crates: js-sys
and web-sys
that already bring all of
common JavaScript API. Let's use them (in Cargo.toml
):
[package]
name = "designer"
version = "0.1.0"
edition = "2021"
[lib]
# I won't use the rlib type
crate-type = ["cdylib"]
[build]
target = "wasm32-unknown-unknown"
[features]
default = ["console_error_panic_hook"]
[dependencies]
[dependencies]
# Updated the versions
wasm-bindgen = "0.2.79"
js-sys = "0.3.56"
# One must point all the needed features, they are on the
# library's documentation, and you know you need one when
# strange "no method named ... found" errors appear for
# things exported by that crate.
web-sys = {version = "0.3.56", features=[
"Element", "HtmlElement", "Document", "DomStringMap",
"HtmlCollection", "DomTokenList", "EventListener",
"MouseEvent", "EventTarget", "console",
]}
# Remove before deployment:
console_error_panic_hook = { version = "0.1.6", optional = true }
# Small but slow memory allocator:
wee_alloc = { version = "0.4.5", optional = true }
[dev-dependencies]
wasm-bindgen-test = "0.3.13"
[profile.release]
# Tell `rustc` to optimize for small code size.
opt-level = "s"
And now I can use the relevant types:
use wasm_bindgen::JsValue;
use web_sys::{HtmlElement, Document};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
/**
* A panel with a companion tab that you can assign
* into tabbed containers, identified by having the
* class="tabspanel" and data-tabsid="centerTabs"
* properties.
*/
struct Tab {
tab: HtmlElement,
panel: HtmlElement,
tab_name: String,
last_focus: Option<String> // Focus history
}
impl Tab{
fn new(tab_name: String, panel: HtmlElement) -> Result<Tab, JsValue> {
// Beware the quotation marks, the error messages for
// missing them are not great.
let doc = Document::new()?;
let tab = doc.create_element("span")?;
// The thing you create with document.createElement
// on javascript isn't garanteed to be an HtmlElement!
let htab = tab.dyn_into::<HtmlElement>()?;
Ok(Tab{
tab: htab, panel: panel,
tab_name: tab_name, last_focus: None
})
}
/// Assigns the tab to that panel
fn assign_container(&self, container: HtmlElement) -> Result<(), JsValue> {
Ok(())
}
/// Selects this tab as the visible one on the container
fn select(&self) -> Result<(), JsValue> {
Ok(())
}
/// Goes back one step on the focus stack
fn unselect(&self) -> Result<(), JsValue> {
Ok(())
}
}
There was no simple element I could use to stub the tab
value on
the creator, so I had to implement this part.
Notice that JavaScript is a dynamic language, and the DOM makes
use of this. In particular, that document.createElement
call can in
theory return many different types of element, not only HTML. That's
the kind of problem this experiment is intended to catch, and
admittedly, that dynamic conversion function wasn't easy to find.
That code is ugly non-idiomatic, but by the documentation, it's the way to actually use that result. But that kind of code can be just encapsulated away.
Anyway, since it's an easy to encapsulate and solved problem, it's not a deal-breaker. Let's move on.
Finally let's export those stubs so they can be used from a web page. For
that, there is the #[wasm_bindgen]
macro:
use std::vec::Vec;
use wasm_bindgen::JsValue;
use web_sys::{HtmlElement, Document};
use wasm_bindgen::JsCast;
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
struct Tab {
tab: HtmlElement,
panel: HtmlElement,
tab_name: String,
last_focus: Vec<String> // Focus history
}
#[wasm_bindgen]
impl Tab{
#[wasm_bindgen(constructor)]
fn new(tab_name: String, panel: HtmlElement) -> Result<Tab, JsValue> {
let doc = Document::new()?;
let tab = doc.create_element("span")?;
let htab = tab.dyn_into::<HtmlElement>()?;
Ok(Tab{tab: htab, panel: panel, tab_name: tab_name, last_focus: Vec::new()})
}
/// Assigns the tab to that panel
fn assign_container(&self, container: HtmlElement) -> Result<(), JsValue> {
Ok(())
}
/// Selects this tab as the visible one on the container
fn select(&self) -> Result<(), JsValue> {
Ok(())
}
/// Goes back one step on the focus stack
fn unselect(&self) -> Result<(), JsValue> {
Ok(())
}
}
That macro has many options you can find on the crate's
documentation. Here I used that constructor
one.
Also, I must reexport the module as public on src/lib.rs
(let's
keep all that hello world stuff for a while longer):
mod utils;
use wasm_bindgen::prelude::*;
// When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
// allocator.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;
pub mod tabs;
#[wasm_bindgen]
extern {
fn alert(s: &str);
}
#[wasm_bindgen]
pub fn greet() {
alert("Hello, {{project-name}}!");
}
Compiling and packaging (with wasm-pack
like done on part 1), when
one inspects the loading script (designer.js
), there is a new class
there, named Tab
. We can now use it in the main JavaScript module
(runwasm.js
):
import init_designer, {Tab} from "./designer.js";
let designer = await init_designer();
let t = new Tab("body", document.body);
designer.greet();
Now it looks like I have every piece needed to implement the entire class in rust. Let's try a direct translation.
The constructor was almost complete already:
fn new(tab_name: &str, panel: HtmlElement) -> Result<Tab, JsValue> {
// Beware the quotation marks, the error messages for
// missing them are not great.
let doc = Document::new()?;
let tab = doc.create_element("span")?;
// The thing you create with document.createElement
// on javascript isn't garanteed to be an HtmlElement!
let htab = tab.dyn_into::<HtmlElement>()?;
htab.set_inner_text(tab_name);
htab.dataset().set("tabname", tab_name);
panel.dataset().set("tabname", tab_name);
Ok(Tab{
tab: htab, panel: panel,
tab_name: tab_name.to_string(), last_focus: None
})
}
Assigning to a container starts easily enough:
/// Assigns the tab to that panel
fn assign_container(&self, container: HtmlElement) -> Result<(), JsValue> {
// Inserts the element
let doc = Document::new()?;
let tabsid = container.dataset().
get("tabsid").ok_or("data-tabsid not found")?;
let tabs = doc.get_element_by_id(&tabsid[..]).
ok_or(format!("Tabs container of id {} not found", tabsid))?;
container.append_with_node_1(&self.panel)?;
But quite soon I have to iterate over HTML elements, and the lifetime analysis conflicts with the dynamic conversion I did earlier. It's not a large problem, since there are all kinds of conversion functions available.
It also didn't save converting the collection into an array for iterating over it. That is something I will have to write in Rust later. Anyway, the goal is to remove as much code as possible from the JavaScript interpreter, so those cases will get rarer once I refactor the code after the migration.
// Selects a tab if none is selected
// This is much worse than the javascript original.
let selected = Array::from(&tabs.children()).iter().filter(
|e| {
match e.dyn_ref::<HtmlElement>() {
None => false,
Some(he) => he.class_list().contains(DISP),
}
}).count();
if selected == 0 {
self.tab.class_list().add_1(DISP);
self.panel.class_list().add_1(DISP);
}
Next comes the really challenging part: assigning event handlers.
The DOM expects JavaScript functions as event handlers. That means whatever function I use I need to import it into JavaScript first, and then assign it in Rust.
You won't see how to do that on the tutorial, or the wasm book. There
is also no direct pointer on the documentation for how to do it. (And
to be honest, the Function
structure of js-sys
is so confusing
that at one point I was ready to declare this experiment a failure.)
But the wasm_bindgen crate has a quite complete guide. There is a section there about closures, that has the correct concept to use here.
Well, I just can't explain how to use it better than the example on the guide. Let's just follow on.
The next problem is that, obviously, that function must now live on the JavaScript interpreter until it gets garbage-collected. That is a different lifetime than the Tab object. On JavaScript, this doesn't look like a problem (it can lead to bugs in theory; in practice, it doesn't), but on Rust, it is.
And it gets worse. Currently there is no way to create such JavaScript
closures from Rust without leaking some memory. Every detail on
wasm_bindgen
is designed expecting static closures, what the one
from the original script is not.
OK, that is bad. Really bad. That does make this translation not advisable on every case.
But specifically on the designer, there aren't many non-static event handlers. So it's viable to convert every one of them to use only global values.
The problem then becomes that Rust has quite bad support for global values...
I need something like this:
static mut tab_registry: HashMap<String, Tab> = HashMap::new();
Except that this doesn't work. One can not create static mutable values.
The global thread-safe primitives also fail to solve the problem. It's not obvious, but their values can not be accessed by the DOM.
There exist thread local values, that have roughly the necessary format (JavaScript couldn't run in more than one thread at all until recently, and the DOM is still completely thread local). But those values must also be read-only.
The solution is contrived, but there is functionality on the standard library for it.
use std;
use std::rc::Rc;
use std::cell::{RefCell, RefMut};
thread_local! (
static TAB_REGISTRY: Rc<RefCell<HashMap<String, Rc<RefCell<Tab>>>>>
= Rc::new(RefCell::new(HashMap::new()));
);
This creates a new modality of run-time errors, that happen if some code that uses the registry calls other code that also tries to use it. So, this solution imposes an specific architecture for the front-end, where there is a global tab manager and only it can access the registry.
That is not a big deal here, because a global tab manager (and global managers for everything) is already on my TODO list. It's way down on my priorities, but it's something I will need.
The handler function, then gets defined this way:
fn select_tab_handler_fn(e: Event) -> Result<(), JsValue> {
let targ_ev = e.target().ok_or("Tab selection event without a target")?;
let targ = targ_ev.dyn_into::<HtmlElement>()?;
let tab_name = targ.dataset().get("tabname")
.ok_or("Tab element does not have a data-tabname property")?;
with_tab_mut(&tab_name, |mut tab| {
tab.select()?;
tab.last_focus = None;
Ok(())
})
}
thread_local! (
static SELECT_TAB_HANDLER: Closure<dyn Fn(Event)> = Closure::wrap(
Box::new(move |e: Event| {
match select_tab_handler_fn(e) {
Ok(_) => (),
Err(e) => console::error(&Array::of1(&e)),
}
}) as Box<dyn Fn(Event)>);
);
But that isn't the only problem with the translation.
In particular, the web-sys
types are mangled in some way and don't
represent exactly the JavaScript types they mean to. As a consequence,
the dynamic type conversions are simply not reliable enough. That
means that part of the web-sys
API is not usable.
It looks like Rust generated wasm is very mature and well tested for the use-case of exporting fast functions from Rust into your JavaScript front-end. But it's not as mature for creating an entire front-end.
Anyway, the bindings themselves are quite mature. It's the imported
API that is not. And the web-sys
API is entirely optional, one can
provide one's own instead.
But let's not be that extreme. A lot of web-sys
works perfectly. I
can replace just the parts that fail. Specifically on this module, the
following is enough:
#[wasm_bindgen]
extern {
#[wasm_bindgen(catch)]
fn create_html(e: &str) -> Result<HtmlElement, JsValue>;
#[wasm_bindgen]
fn html_by_id(i: &str) -> HtmlElement;
#[wasm_bindgen]
fn has_class(e: &JsValue, class: &str) -> bool;
#[wasm_bindgen(catch)]
fn set_class(e: &JsValue, class: &str) -> Result<(), JsValue>;
#[wasm_bindgen(catch)]
fn clear_class(e: &JsValue, class: &str) -> Result<(), JsValue>;
#[wasm_bindgen(catch)]
fn set_data(e: &JsValue, key: &str, value: &str) -> Result<(), JsValue>;
#[wasm_bindgen]
fn get_data(e: &JsValue, key: &str) -> String;
#[wasm_bindgen(catch)]
fn set_onclick(e: &JsValue, on_click: &Function) -> Result<(), JsValue>;
}
The JavaScript implementation of those functions is trivial, and they
are well defined enough to never change, so they are not a maintenance
burden. They just have to be defined on the main JavaScript namespace
(or you can use the namespace
option or wasm_bindgen
), so they can
not be on a module.
Also, the correct mutability, adequate Option
return values and
specific parameter types all need some encapsulation and private code.
As a consequence, they are better done in a separated module.
For the duration of this experiment, I will let those incorrect. I will fix them when I create an architecture for the entire front-end.
And that's it. Really. After fixing those issues, things work. Now you go to the conclusion, on the third part of this article, or you can go read the first part if you want to go back.
The Designer is a database designing tool that is currently under development.
It's intended to integrate well with git and a git-based workflow, while also being able to replace the database access applications during development, allowing for the management of the data and schema migration right from the same place.
Currently, the Designer is written in vanilla web languages.
That wasn't a conscious choice. Instead the current format of the editor is the consequences of a series of experiments on laying-out the desired interaction on a web page with modern browser features.
And my conclusion form those experiments is that the modern web is going in a pretty good direction!
CSS is getting great, and the DOM is quite expressive nowadays. Doing things in vanilla web tech has never been so easy... and I'll just leave JavaScript out of the conclusion.
I started those experiments expecting JavaScript to break at some point. As expected, it did, but in an unexpected way. Now that it is broken and I know how, it's time to fix things.
This is a multi-part article. Here at the first part we will get the problem description and a hello world wasm project working.
The second part has an attempted solution and the description of the problems found integrating Rust generated wasm with the DOM.
The third part has the conclusion and the final code.
By the way, WASM here is the acronym form Web Assembly.
Nowadays every browser comes with a builtin virtual machine that can execute compiled programs instead of JavaScript. The binary format of those programs is called Web Assembly.
OK, here is the problem, I've made about 10% of the way of my first release's TODO list, and most of the code modules and structure that will ever be there are already created. The next 90% of the work consists on changing that code.
JavaScript beared writing the code much better than expected. There are very good idioms one can adopt on modern JavaScript. There are some language inconsistencies and footguns, but they aren't a overwhelming problem.
It's not writing code that is the issue. The problem is that changing code is much harder than in most languages.
And by "much" I mean really much.
Everything points that if I insist on vanilla JavaScript, I will end with testing paralysis, where fixing broken tests will clearly become my dominant time sink.
So, I must use something else. And that something else avoid the need for most tests.
The one thing that will solve my problem is a good static and strict type system. And the obvious way to get that for the web is by migrating into Typescript.
Knwoing that Typescript would work, I don't need an experiment for it. Instead, I will try an uncertain option that is expected to eliminate the JavaScript annoyances (as much as they are less annoying than I expected, they are still annoying) and, since diagramming things is full of integer optimization problems, bring better CPU usage and native support for those integer values I'll optimize.
Let's pick the simplest JavaScript module I have on the designer code. This creates a tabbed panel that one can assign into containers:
const disp = 'display';
export class Tab {
constructor(name, panel){
let tab = document.createElement('span');
tab.innerText = name;
this.tab = tab;
tab.dataset.tabname = name;
this.panel = panel;
panel.dataset.tabname = name;
this.lastFocus = null;
}
/// Assigns the tab to that panel
assignContainer(container) {
// Inserts the elements
const tabs = document.getElementById(container.dataset.tabsid);
tabs.append(this.tab);
container.append(this.panel);
// If no tab is selected, select this
const d = Array.from(tabs.children).
filter((x) => x.classList.contains(disp));
if(d.length == 0) {
this.tab.classList.add(disp);
this.panel.classList.add(disp);
}
// Selection handling
this.tab.onclick = ((e) => {
this.select();
this.lastFocus = null;
});
}
// Selects this tab on the container that contains it
select() {
let sel = Array.from(this.tab.parentElement.children).
filter(x => x.classList.contains(disp));
if(sel[0]) {
this.lastFocus = sel[0].dataset.tabname;
}
Array.from(this.tab.parentElement.children).
forEach((e) => e.classList.remove(disp));
this.tab.classList.add(disp);
Array.from(this.panel.parentElement.children).
forEach((e) => e.classList.remove(disp));
this.panel.classList.add(disp);
}
// Goes back one step on the focus stack
unselect() {
if(this.lastFocus) {
let o = this.lastFocus;
Array.from(this.tab.parentElement.children).
forEach(
(e) =>
e.dataset.tabname != o? e.classList.remove(disp):
e.classList.add(disp)
);
Array.from(this.panel.parentElement.children).
forEach(
(e) =>
e.dataset.tabname != o? e.classList.remove(disp):
e.classList.add(disp)
);
this.lastFocus = null;
}
}
}
This is a very small module, that makes almost no processing and heavily uses the DOM interface. Or, in other words, this is a low-risk module for experimenting that is also a worst case for wasm translation.
It could use some improvements too. Both some low-gain self-contained ones that won't break anything and some high-gain ones that have a good chance of impacting other modules. Doing any of them is out of the scope here.
On the Rust front, things have evolved a lot since the early days of wasm.
Not only cross-compiling Rust got easier, there are complete abstraction layers one can use nowadays, and a lot of documentation. There is even an official book on the Rust site teaching how do wasm. It's not something that takes research anymore.
The documentation still isn't perfect. All of it suffers from the same lack of discoverability that is normal for Rust. But for sure, it looks complete.
So, let's get to it.
First steps, the site's tutorial uses a prebuilt template, so I'll take the important settings from it and get a working hello world application.
Creating the project...
$ cargo init designer
On the Cargo.toml
, it uses the following:
[package]
name = "designer"
version = "0.1.0"
edition = "2021"
# Bellow this point, all comes directly from the WASM template
[lib]
crate-type = ["cdylib", "rlib"]
[features]
default = ["console_error_panic_hook"]
[dependencies]
wasm-bindgen = "0.2.63"
# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
# all the `std::fmt` and `std::panicking` infrastructure, so isn't great for
# code size when deploying.
console_error_panic_hook = { version = "0.1.6", optional = true }
# `wee_alloc` is a tiny allocator for wasm that is only ~1K in code size
# compared to the default allocator's ~10K. It is slower than the default
# allocator, however.
#
# Unfortunately, `wee_alloc` requires nightly Rust when targeting wasm for now.
wee_alloc = { version = "0.4.5", optional = true }
[dev-dependencies]
wasm-bindgen-test = "0.3.13"
[profile.release]
# Tell `rustc` to optimize for small code size.
opt-level = "s"
OK, if you have cross-compiled Rust before, it is surprising that there
isn't any .cargo
directory on the template. That's because the wasm
is linked by a specialized tool.
That wasm-bindgen
dependency does the FFI bindings for any
JavaScript API into Rust and the Rust API into JavaScript...
Bingo! I got an entire row of acronyms!
Let expand them here on the case you never heard one or another:
FFI: Foreign Function Interface. That's how you call a function from one language in another language. A FFI bindings library has some native functions (here, Rust functions) that do nothing but call foreign functions (here, JavaScript ones).
API: Application Programming Interface. An interface that programs use, instead of people. Here, it's a set of functions.
Thus, that line means it's a library that allow you to run the relevant JavaScript functions in Rust or the Rust ones in JavaScript.
... And there's some functionality for displaying better error messages. Rust people do love their error messages (rightly so). Also there are settings that reduce the binary size, what is quite important on the web.
The template also has a file at src/util.rs
with a function that you
have to call to actually get the improved errors, and a hello world in
src/lib.rs
. I will just copy both verbatim.
For building, I'll need the wasm target:
$ rustup target add wasm32-unknown-unknown
And you can build for the correct architecture, link and pack everything on the same step:
$ cargo install wasm-pack # If you don't have it
$ wasm-pack build --target web
As that finished, it created a pkg
directory inside the project that contains
all the relevant files for npm to require it. And then comes a real
bunch of magic.
The book and the tutorial instruct one into using npm and web-pack to use those files. But since the idea is getting ride of JavaScript annoyances, let's take a deeper look at them.
Looking at the generated files, one can see that most are
about the npm project, Typescript interface, and other
kinds of integration. The actually important ones are the wasm file (on my
case designer_bg.wasm
, no idea what that bg
means) and the
JavaScript loader (for me, designer.js
).
The wasm is loaded as a module. And since it doesn't execute any code (only exports some), it must be imported by another module. On the interface, there is a default export that initializes the wasm code. So, to run the hello world, I will need a module like this:
import init_designer from "./designer.js";
let designer = await init_designer();
// Greet comes all the way form the lib.rs interface
designer.greet();
You can read about async-await at MDN.
The Rust primitives for concurrency are much harder to use wrongly, but async-await makes JavaScript about as usable even if error-prone.
There is actually a way to embedded code as executable on the loader, but it must be initialized anyway, so it can not be imported directly from the HTML.
Now I have an executable module that I can import into the HTML
(considering it's named runwasm.js
):
<!DOCTYPE html>
<script type="module" src="runwasm.js"></script>
But wait, I just loaded the HTML file, to import an executable JavaScript module, to import the wasm loader, to load the wasm. No wonder the docs go directly for web-pack, those are a lot of consecutive network accesses.
But I can also avoid the problem by prefetching everything, JavaScript tooling is not a hard requirement. For the designer it won't make any difference for a long time, because the first version is supposed to run from localhost, but it's a good thing to do anyway.
<!DOCTYPE html>
<head>
<link rel="preload" href="/designer.js" as="script">
<!-- That crossorigin is required, but not relevant here -->
<link rel="preload" href="/designer_bg.wasm" as="fetch" crossorigin="anonymous">
</head>
<script type="module" src="runwasm.js"></script>
And well, that was easier than expected! Launch a local HTTP server where everything is located, like with:
$ touch favicon.ico # So we don't get 404 errors
$ python3 -m http.server
And it's alive!
Beware that the pkg
directory is removed every time you build your
project, so it you want to try it, it's better to move or link the
files in a different one.
Next, on the second part there is some Rust that actually does something and problems that appeared.
Finally, on the third part there is the final version of the code.
The Designer is an entity-relationship model (ERM) viewer and editor, aimed at designing physical databases (only databases, and only physical). It is currently under development and will probably be the first component you can actually download from here and use.
The designer will be a desktop and web (on premises) application, that is aimed at both graphically designing your data and support all the common database activities one does during development, like creating test data, editing domain data, visualizing the database state, or verifying SQL code.
It is also aimed at database synchronization and also script creation and management through a deployment pipeline.
The designer UX is based on a git development model, so its data is stored on legible text files (in a language that is being developed, to be announced), structured in a way that minimizes diffs. It will also support graphical diffs and diff3 merging, so that the Designer supports the entire life cycle of a git project.
Features for design reuse, automation and verification are also on the planning. But those will come later on the development, so their exact format is still subject to changes.
Anyway, that is what is planned. Currently the Designer is a very basic web-based modeler, with the minimum functionality still unfinished.
It currently looks like this:
There is ongoing work both on the front and back ends. I plan to make a first release as soon as the ERM editor is complete, so you can try it.
At first it will only have the web front end, that should run on a local computer to edit on-disk diagram files. It will still not support any of the features from the plan up there, not even the high priority ones for interacting with databases and git. Those should come shortly after.
And, well, that's it. Back to programming now. I hope to post more news here soon.