1. DRAFT dylib-skel-1
- State "DRAFT" from
1.1. Overview
Our core languages are Rust and Lisp - this is the killer combo which will allow NAS-T to rapidly develop high-quality software. As such, it's crucial that these two very different languages (i.e. compilers) are able to interoperate seamlessly.
Some interop methods are easy to accomodate via the OS - such as IPC or data sharing, but others are a bit more difficult.
In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something that can be difficult, due to some complications with Rust and because this is not the most popular software stack (yet ;). This is an experiment and may not make it to our code-base, but it's definitely something worth adding to the toolbox in case we need it.
1.2. FFI
The level of interop we're after in this case is FFI.
Basically, calling Rust code from Lisp and vice-versa. There's an article about calling Rust from Common Lisp here which shows the basics and serves as a great starting point for those interested.
1.2.1. Rust != C
The complication(s) with Rust I mentioned early is really just that it is not C. C
is old, i.e. well-supported with a stable ABI, making the process of creating bindings
for a C library a breeze in many languages.
For a Rust library we need to first appease the compiler, as explained in this section of the Rustonomicon. Among other things it involves changing the calling-convention of functions with a type signature and editing the Cargo.toml file to produce a C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used like the C ABI can.
1.2.2. Overhead
Using FFI involves some overhead. Check here for an example benchmark across a few languages. While building the NAS-T core, I'm very much aware of this, and will need a few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular, I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
1.3. Rust -> C -> Lisp
1.3.1. Setup
For starters, I'm going to assume we all have Rust (via rustup
) and Lisp (sbcl
only)
installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
this post).
- Cargo
Create a new library crate. For this example we're focusing on a 'skeleton' for dynamic libraries only, so our experiment will be called
dylib-skel
or dysk for short.cargo init dysk --lib && cd dysk
A
src/lib.rs
will be generated for you. Go ahead and delete that. We're going to be making our ownlib.rs
file directly in the root directory (just to be cool).The next step is to edit your
Cargo.toml
file. Add these lines after the[package]
section and before[dependencies]
:[lib] crate-type = ["cdylib","rlib"] path = "lib.rs" [[bin]] name="dysk-test" path="test.rs"
This tells Rust to generate a shared C-compatible object with a
.so
extension which we can open using dlopen. - cbindgen
- install
Next, we want the
cbindgen
program which we'll use to generate header files for C/C++. This step isn't necessary at all, we just want it for further experimentation.cargo install --force cbindgen
We append the
cbindgen
crate as a build dependency to ourCargo.toml
like so:[build-dependencies] cbindgen = "0.24"
- cbindgen.toml
language = "C" autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" include_version = true namespace = "dysk" cpp_compat = true after_includes = "#define DYSK_VERSION \"0.1.0\"" line_length = 88 tab_width = 2 documentation = true documentation_style = "c99" usize_is_size_t = true [cython] header = '"dysk.h"'
- build.rs
fn main() -> Result<(), cbindgen::Error> { if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) { b.write_to_file("dysk.h"); Ok(())} else { panic!("failed to generate dysk.h from cbindgen.toml") } }
- install
1.3.2. lib.rs
//! lib.rs --- dysk library use std::ffi::{c_char, c_int, CString}; #[no_mangle] pub extern "C" fn dysk_hello() -> *const c_char { CString::new("hello from rust").unwrap().into_raw()} #[no_mangle] pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b} #[no_mangle] pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
1.3.3. test.rs
//! test.rs --- dysk test fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
1.3.4. compile
cargo build --release
1.3.5. load from SBCL
(load-shared-object #P"target/release/libdysk.so") (define-alien-routine dysk-hello c-string) (define-alien-routine dysk-plus int (a int) (b int)) (define-alien-routine dysk-plus1 int (n int)) (dysk-hello) ;; => "hello from rust"
1.3.6. benchmark
time target/release/dysk-test
(time (dotimes (_ 500000000) (dysk-plus1 2)))