Shared Library Skeletons
1. Overview
- CODE
- packy/stash/dysk
Our core languages are Rust and Lisp - this is the killer combo which will allow NAS-T to rapidly develop high-quality software. As such, it's crucial that these two very different languages (i.e. compilers) are able to interoperate seamlessly.
Some interop methods are easy to accomodate via the OS - such as IPC or data sharing, but others are a bit more difficult.
In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something that can be difficult, due to some complications with Rust and because this is not the most popular software stack (yet ;). This is an experiment and may not make it to our code-base, but it's definitely something worth adding to the toolbox in case we need it.
1.1. FFI
The level of interop we're after in this case is FFI.
Basically, calling Rust code from Lisp and vice-versa. There's an article about calling Rust from Common Lisp here which shows the basics and serves as a great starting point for those interested.
1.1.1. Rust ABI
The complication(s) with Rust I mentioned early is really just that it is not C. C
is old, i.e. well-supported with a stable ABI, making the process of creating bindings
for a C library a breeze in many languages.
For a Rust library we need to first appease the compiler, as explained in this section of the Rustonomicon. Among other things it involves changing the calling-convention of functions with a type signature and editing the Cargo.toml file to produce a C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used like the C ABI can.
abi_stable_crates is a project which addresses some of the ABI concerns, presenting a sort of ABI-API as a Rust library. Perhaps this is the direction the ecosystem will go with in order to maintain an unstable ABI, but for now there is no 'clear' pathway for a friction-less FFI development experience in Rust.
1.1.2. Overhead
Using FFI involves some overhead. Check here for an example benchmark across a few languages. While building the NAS-T core, I'm very much aware of this, and will need a few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular, I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
2. basic example
2.1. Setup
For starters, I'm going to assume we all have Rust (via rustup) and Lisp (sbcl only) installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in this post).
2.1.1. Cargo
Create a new library crate. For this example we're focusing on a 'skeleton' for
dynamic libraries only, so our experiment will be called dylib-skel
or dysk for
short.
cargo init dysk --lib && cd dysk
A src/lib.rs
will be generated for you. Go ahead and delete that. We're going to be
making our own lib.rs
file directly in the root directory (just to be cool).
The next step is to edit your Cargo.toml
file. Add these lines after the [package]
section and before [dependencies]
:
[lib] crate-type = ["cdylib","rlib"] path = "lib.rs" [[bin]] name="dysk-test" path="test.rs"
This tells Rust to generate a shared C-compatible object with a .so
extension which we
can open using dlopen.
2.1.2. cbindgen
- install
Next, we want the
cbindgen
program which we'll use to generate header files for C/C++. This step isn't necessary at all, we just want it for further experimentation.cargo install --force cbindgen
We append the
cbindgen
crate as a build dependency to ourCargo.toml
like so:[build-dependencies] cbindgen = "0.24"
- cbindgen.toml
language = "C" autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */" include_version = true namespace = "dysk" cpp_compat = true after_includes = "#define DYSK_VERSION \"0.1.0\"" line_length = 88 tab_width = 2 documentation = true documentation_style = "c99" usize_is_size_t = true [cython] header = "dysk.h"
- build.rs
fn main() -> Result<(), cbindgen::Error> { if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) { b.write_to_file("dysk.h"); Ok(())} else { panic!("failed to generate dysk.h from cbindgen.toml") } }
2.2. lib.rs
//! lib.rs --- dysk library use std::ffi::{c_char, c_int, CString}; #[no_mangle] pub extern "C" fn hello() -> *const c_char { CString::new("hello from rust").unwrap().into_raw()} #[no_mangle] pub extern "C" fn plus(a:c_int,b:c_int) -> c_int {a+b} #[no_mangle] pub extern "C" fn plus1(n:c_int) -> c_int {n+1}
2.3. test.rs
//! test.rs --- dysk test fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::plus1(2 as core::ffi::c_int);}}
2.4. compile
cargo build --release
2.5. load from SBCL
;;; dysk.lisp ;; (dysk:hello) ;; => "hello from rust" (defpackage :dysk (:use :cl :sb-alien) (:export :hello :plus :plus1)) (in-package :dysk) (load-shared-object #P"target/release/libdysk.so") (define-alien-routine hello c-string) (define-alien-routine plus int (a int) (b int)) (define-alien-routine plus1 int (n int))
2.6. benchmark
time target/release/dysk-test
(time (dotimes (_ 500000000) (dysk:plus1 2)))