An SML Rabbit Hole

Since I've been having fun exploring SML and SMLNJ in particular I was recently trying (and failing!) to explore avenues for integrating this neat little language with other systems. In particular I was keen to try out some basic C interfacing with the project's "No-Longer-Foreign Function Interface" because it is well documented and there's some precedent for me being able to wrangle some exceedingly modest FFI as I explore new languages.

My initial motivation was probably too ambitious so I'm making some notes on what I've run into along the way.

NLFFI is not supported on 64-bit builds. This wasn't enough to convince me to stop trying so I switched to the recommended "legacy" repo and found the necessay flag to the build script is: config/install.sh -default 32

The documentation above does call out:

Missing from our type encoding is a way of fully handling variable-argument functions such as C’s printf, a shortcoming that we intend to address soon.

And it appears support is still missing. So rather than re-use my previous FFI example exactly to print some useless text I figured I could settle for an example from the paper:

sqrt(3)                                Library Functions Manual                               sqrt(3)

NAME
       sqrt, sqrtf, sqrtl - square root function

LIBRARY
       Math library (libm, -lm)

SYNOPSIS
       #include <math.h>

       double sqrt(double x);
       float sqrtf(float x);
       long double sqrtl(long double x);

Next I ran into an issue of type support, also called out in the paper. From the man page above we see that math.h implements versions of square root for double, float, and long double. The linked paper says:

SML/NJ currently does not have the full variety of precisions for integral and floating-point types that a typical C compiler would provide.

And we find that the ml-nlffi is unhappy to ingest a header with a long double (even if we aren't going to use it!).

$ echo '#include <math.h>' > uses-math.h

$ ml-nlffigen -match '.*math.*' -dir gen -cmfile m.cm uses-math.h
"/usr/include/math.h":423.1-440.4: warning: trailing comma in enum declaration
Fail: tyConId
	hash.sml:32.27-32.41

I'll give it away a bit and tell you that the warning doesn't seem to matter much and the issue is really the mysterious failure message. A bit of trial and error binary search winnows things down to something like this:

$ echo 'long double sqrt(long double x);' > ld.h

$ ml-nlffigen -dir gen -cmfile ld.cm ld.h
Fail: tyConId
	hash.sml:32.27-32.41

$ echo 'double sqrt(double x);' > ld.h
$ ml-nlffigen -dir gen -cmfile ld.cm ld.h
$ echo $?
0

$ echo 'long sqrt(long x);' > ld.h
$ ml-nlffigen -dir gen -cmfile ld.cm ld.h
$ echo $?
0

Not to be deterred though, this just means I need my own header file to declare the function and skip any handling of those problem types.

$ cat sqrt.h
double sqrt(double x);

Trying to following the paper to connect the generated code (ml-nlffigen is a code generator) for the header with the shared object that implements it proved a little interesting because the paper uses a functor but then mention the implementation isn't exact. It seems like the design was changed and ml-nlffigen doesn't use functors (any more?). If instead we root around in the documentation (nlffi/gen/README):

  To actually implement FooH.libh, use the "DynLinkage" module.
  Suppose the shared library's name is "/usr/lib/foo.so".  Here is
  the corresponding contents of foo-h.sml:

  +-------------------------------------------------------------
  |structure FooH = struct
  |    local 
  |        val lh = DynLinkage.open_lib
  |             { name = "/usr/lib/foo.so", global = true, lazy = true }
  |    in
  |        fun libh s = let
  |            val sh = DynLinkage.lib_symbol (lh, s)
  |        in
  |            fn () => DynLinkage.addr sh
  |        end
  |    end
  |end
  +-------------------------------------------------------------

  If all the symbols you are linking to are already available within
  the ML runtime system, then you don't need to open a new shared
  object.  As a result, your FooH implementation would look like this:

  +-------------------------------------------------------------
  |structure FooH = struct
  |    fun libh s = let
  |        val sh = DynLinkage.lib_symbol (DynLinkage.main_lib, s)
  |    in
  |        fn () => DynLinkage.addr sh
  |    end
  |end
  +-------------------------------------------------------------
  

Following the documentation instead isn't especially surprising or difficult and results in:

structure LibmH = struct
    local 
        val lh = DynLinkage.open_lib
             { name = "libm.so.6", global = true, lazy = true }
    in
        fun libh s = let
            val sh = DynLinkage.lib_symbol (lh, s)
        in
            fn () => DynLinkage.addr sh
        end
    end
end

$ tree
.
├── libm-h.sml
└── sqrt.h

It is possible to generate the code to interface the C function sqrt into SML with the following: ml-nlffigen -include ../libm-h.sml -libhandle LibmH.libh -dir gen -cmfile sqrt.cm sqrt.h

A slightly annoying quirk (also called out in the documentation) is that the include is supposed to be the path that gets included in the generated subdirectory, so despite the file libm-h.sml being in the present directory, we include with the upper directory accessor. The result looks like this:

$ tree
.
├── gen
│   ├── callop-0.sml
│   ├── fptr-rtti-0.sml
│   ├── f-sqrt.sml
│   └── sqrt.cm
├── libm-h.sml
├── sqrt.h
└── repro.sml

and finally using it looks like this:

$ sml
Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
- CM.make "$c/c.cm";
[autoloading]
[library $smlnj/cm/cm.cm is stable]
[library $smlnj/internal/cm-sig-lib.cm is stable]
[library $/pgraph.cm is stable]
[library $smlnj/internal/srcpath-lib.cm is stable]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[library $c/c.cm is stable]
[library $c/internals/c-int.cm is stable]
[library $c/memory/memory.cm is stable]
[New bindings added.]
val it = true : bool
- CM.make "gen/sqrt.cm";
[scanning gen/sqrt.cm]
[parsing gen/(sqrt.cm):f-sqrt.sml]
[creating directory gen/.cm/SKEL]
[parsing gen/(sqrt.cm):callop-0.sml]
[parsing gen/(sqrt.cm):fptr-rtti-0.sml]
[loading gen/(sqrt.cm):../libm-h.sml]
[compiling gen/(sqrt.cm):callop-0.sml]
[creating directory gen/.cm/x86-unix]
[code: 300, env: 415 bytes]
[compiling gen/(sqrt.cm):fptr-rtti-0.sml]
[code: 568, env: 229 bytes]
[compiling gen/(sqrt.cm):f-sqrt.sml]
[code: 1049, data: 33, env: 274 bytes]
$Execute: required privileges are:
  c-int
  primitive
[New bindings added.]
val it = true : bool
- val root = F_sqrt.f' 2.0;
val root = 1.41421356237 : real

We Couldn't Just Stop There Though

All of the above is mostly unsurprising and works as described. Unfortunately that isn't exactly what I was trying to do in the first place. As mentioned, I was trying to do something slightly more ambitious to begin with and when that kept going awry I peeled back layer after layer until I had the above which was working and a slightly more involved program that was not.

Without getting sidetracked with too much detail I was trying to interface with SQLite and while it appeared to be working briefly I kept hitting a weird error that seemed unrelated to SQLite or my program text. Rather than introduce a bunch of complexity around SQLite though I was able to pare it back to the above C function for square root (nearly the most uninteresting C FFI I can imagine) and a program that doesn't really do anything except waste some time.

I'll admit I didn't come up with this signal handling and interval timer logic, it is lifted more or less directly out of CML (Concurrent ML) which is where I started running into problems. Rather than introduce that complexity either let's just say I have a program that starts, registers a 10ms check on sigALRM with which it does nothing, waits for 1 second, prints, and then exits.

val _ = Signals.setHandler (Signals.sigALRM, Signals.HANDLER (fn (_, _, k) => k));

val () = SMLofNJ.IntervalTimer.setIntTimer (SOME (Time.fromMilliseconds 10));

val stop = Time.+ (Time.now (), Time.fromMilliseconds 1000);

fun wait () = if Time.< (Time.now (), stop) then wait () else ();

val () = wait ();

val () = print "finished\n";
val () = OS.Process.exit OS.Process.success;

Running it looks like this:

$ sml wait-and-exit.sml 
Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
[opening wait-and-exit.sml]
[autoloading]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[autoloading]
[autoloading done]
val stop = TIME {usec=1791059099277510} : Time.time
val wait = fn : unit -> unit
finished

$ echo $?
0

Totally unsurprising, I think. Except for the explicit interest registered on the alarm signal it is functionally a program that sleeps 1 second.

But if we introduce our working FFI call, even though it does nothing with the rest of the program:

CM.make "$c/c.cm";
CM.make "gen/sqrt.cm";

val root = F_sqrt.f' 2.0;
val () = print ("sqrt 2.0 = " ^ Real.toString root ^ "\n");

val _ = Signals.setHandler (Signals.sigALRM, Signals.HANDLER (fn (_, _, k) => k));

val () = SMLofNJ.IntervalTimer.setIntTimer (SOME (Time.fromMilliseconds 10));

val stop = Time.+ (Time.now (), Time.fromMilliseconds 1000);

fun wait () = if Time.< (Time.now (), stop) then wait () else ();

val () = wait ();

val () = print "finished\n";
val () = OS.Process.exit OS.Process.success;

It starts doing something weird:

$ sml repro.sml 
Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
[opening repro.sml]
[autoloading]
[library $smlnj/cm/cm.cm is stable]
[library $smlnj/internal/cm-sig-lib.cm is stable]
[library $/pgraph.cm is stable]
[library $smlnj/internal/srcpath-lib.cm is stable]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[library $c/c.cm is stable]
[library $c/internals/c-int.cm is stable]
[library $c/memory/memory.cm is stable]
[New bindings added.]
val it = true : bool
[scanning gen/sqrt.cm]
[loading gen/(sqrt.cm):../libm-h.sml]
[loading gen/(sqrt.cm):callop-0.sml]
[loading gen/(sqrt.cm):fptr-rtti-0.sml]
[loading gen/(sqrt.cm):f-sqrt.sml]
$Execute: required privileges are:
  c-int
  primitive
[New bindings added.]
val it = true : bool
val root = 1.41421356237 : real
[autoloading]
[autoloading done]
sqrt 2.0 = 1.41421356237
[autoloading]
[autoloading done]
val stop = TIME {usec=1791059444664596} : Time.time
val wait = fn : unit -> unit
/home/nolan/others/legacy/bin/sml: Fatal error -- Uncaught exception Fail with "inconsistent state DEFAULT for signal 21" raised at Basis/Implementation/NJ/internal-signals.sml:346.16-346.24

I can't describe how entirely puzzling this was when I got the program narrowed down this far. What the heck is that? I actually spent longer still making sure that it wasn't my printing the result of the C call that was causing issues. In the documentation I quoted above it mentions you can get a shorter library declaration for things already included in the runtime which the math library already is. I tried leaving out as much as I could until I got it down to just using the C FFI with no library to invoke free(null) which this StackOverflow answer highlights as being spec-defined as a no-op.

And STILL the problem remains.

Archaeology

So what the heck is it? Following the cited line in the exception leads you to the signal handling. My reading of that exception is that the signal table at the index position of "code" (21) was not matched by a signal handler with "mask = 0". But what does that mean? What is signal 21? man 7 signal:

   Signal numbering for standard signals
       The numeric value for each signal is given in the table below.  As shown in the table, many signals have
       different  numeric  values  on different architectures.  The first numeric value in each table row shows
       the signal number on x86, ARM, and most other architectures; the second value is for  Alpha  and  SPARC;
       the  third  is  for MIPS; and the last is for PARISC.  A dash (-) denotes that a signal is absent on the
       corresponding architecture.
       Signal        x86/ARM     Alpha/   MIPS   PARISC   Notes
                   most others   SPARC
       ─────────────────────────────────────────────────────────────────
       SIGHUP           1           1       1       1
       SIGINT           2           2       2       2
       SIGQUIT          3           3       3       3
       SIGILL           4           4       4       4
       SIGTRAP          5           5       5       5
       SIGABRT          6           6       6       6
       SIGIOT           6           6       6       6
       SIGBUS           7          10      10      10
       SIGEMT           -           7       7      -
       SIGFPE           8           8       8       8
       SIGKILL          9           9       9       9
       SIGUSR1         10          30      16      16
       SIGSEGV         11          11      11      11
       SIGUSR2         12          31      17      17
       SIGPIPE         13          13      13      13
       SIGALRM         14          14      14      14
       SIGTERM         15          15      15      15
       SIGSTKFLT       16          -       -        7
       SIGCHLD         17          20      18      18
       SIGCLD           -          -       18      -
       SIGCONT         18          19      25      26
       SIGSTOP         19          17      23      24
       SIGTSTP         20          18      24      25
       SIGTTIN         21          21      26      27
       SIGTTOU         22          22      27      28
       SIGURG          23          16      21      29
       SIGXCPU         24          24      30      12
       SIGXFSZ         25          25      31      30
       SIGVTALRM       26          26      28      20
       SIGPROF         27          27      29      21
       SIGWINCH        28          28      20      23
       SIGIO           29          23      22      22
       SIGPOLL                                            Same as SIGIO
       SIGPWR          30         29/-     19      19
       SIGINFO          -         29/-     -       -
       SIGLOST          -         -/29     -       -
       SIGSYS          31          12      12      31
       SIGUNUSED       31          -       -       31

SIGTTIN... job control!?

After a lot more digging, no, not job control.

Poring over the NLFFI doc again turns up this description of signal handling:

5.5.2 Efficient signal handling

When a SML/NJ program is interrupted by an asynchronous signal, then execution must first advance to the next safe point before an ML signal handler can be invoked. Low-level signal handlers (which are part of the C runtime system) record the arrival of a signal and code generated by the ML compiler checks for this condition at regular intervals. A popular technique that eliminates extra runtime overhead for signal polling is to make the heap-limit check do double duty: The C handler records the arrival of a signal by setting the current heap-limit to zero. This causes the next heap-limit check to fail, and subsequent tests (which are no longer on the critical path) can then distinguish between genuine heap overflows and signals.

But the heap-limit is often implemented as a register. Blindly setting this register to zero while anything but ML code is executing is dangerous. Therefore, setting up a C call from ML involves temporarily turning off this form of signal handling. In SML/NJ, this is done by setting the inML-flag to 0 before the call and back to 1 after its return. The old FFI avoids losing signals by branching into a special runtime routine after returning from the C call. The routine checks for interrupts that may have arrived while signal handling was suspended. This technique is safe but expensive.

Our new implementation avoids much of the runtime penalty because it does not need to check for pending signals explicitly and can fully rely on the next heap- limit check: Before performing a C call, a few instructions of in-line code first set the (new) limitPtrMask to the all-ones bit pattern and inML to 0. After the call returns, inML is restored to 1. A final instruction then atomically performs:

limitPtr <- bitwise-and (limitPtr, limitPtrMask)

The low-level signal handler stores 0 into limitPtr (as before) if inML is set but also stores 0 into limitPtrMask regardless of the state of inML. This arrangement guarantees that any signal eventually causes limitPtr to be 0 no matter when it arrives. The atomicity of the bit operation is key to avoiding races.

So we get to go spelunking into the runtime code. The names in the doc are still relevant with the implementation which makes thing easy, grep around for "limitPtr" and "limitPtrMask". The code that relates to the cited paragraphs above seems to begin around here:

...

           val LimitPtrMask =
               M.LOAD (32, ea (vsp', MS.LimitPtrMaskOffVSP),
                       R.memory)
       in
           (* move vsp to its register *)
           emit (assign (vsp', vsp));
           f { inML = inML, LimitPtrMask = LimitPtrMask }
       end (* withVSP *)

       (* prepare for leaving ML *)
       val () =
       withVSP (fn { inML, LimitPtrMask } =>
                   ((* set vp_limitPtrMask to ~1 *)
                    emit (assign (LimitPtrMask, LW 0wxffffffff));
                    (* set vp_inML to 0 *)
                    emit (assign (inML, LW 0w0))));

       (* now do the actual call! *)
       val () =
       app emit callseq;

       (* come back to ML, restore proper limit pointer *)
       val () =
       withVSP (fn { inML, LimitPtrMask } =>
                   ((* set vp_inML back to 1 *)
                    emit (assign (inML, LW 0w1));
                    (* limitPtr := limitPtr & vp_limitPtrMask *)
                    emit (assign (C.limitptr(vfp),
                                  M.ANDB (pty, LimitPtrMask,
                                               C.limitptr(vfp))))));

Looking around the C code for signal handling I turned up a likely looking ifdef which, when you compile a runtime with it, produces some additional logging:

Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
[opening repro.sml]
[autoloading]
[library $smlnj/cm/cm.cm is stable]
[library $smlnj/internal/cm-sig-lib.cm is stable]
[library $/pgraph.cm is stable]
[library $smlnj/internal/srcpath-lib.cm is stable]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[library $c/c.cm is stable]
[library $c/internals/c-int.cm is stable]
[library $c/memory/memory.cm is stable]
[New bindings added.]
val it = true : bool
[scanning gen/sqrt.cm]
[loading gen/(sqrt.cm):../libm-h.sml]
[loading gen/(sqrt.cm):callop-0.sml]
[loading gen/(sqrt.cm):fptr-rtti-0.sml]
[loading gen/(sqrt.cm):f-sqrt.sml]
$Execute: required privileges are:
  c-int
  primitive
[New bindings added.]
val it = true : bool
val root = 1.41421356237 : real
[autoloading]
[autoloading done]
sqrt 2.0 = 1.41421356237
[autoloading]
[autoloading done]
val stop = TIME {usec=1791568844229983} : Time.time
val wait = fn : unit -> unit

CSigHandler[1]: sig = 14, inML = 0, pending = 0, inHandler = 0
RunML: handler pending
ChooseSignal: sig = 14, count = 1
MakeHandlerArg: resumeC = 0xf7b64a18, arg = 0xf7b64a44
REQ_SIG_RETURN: arg = 0xf6796688, pending = 0, inHandler = 1, nSigs = 1/1
REQ_SIG_RESUME: arg = 0x1
LoadResumeState:

CSigHandler[1]: sig = 14, inML = 1, pending = 0, inHandler = 0
RunML: handler pending
ChooseSignal: sig = 21, count = 1
MakeHandlerArg: resumeC = 0xf7b8371c, arg = 0xf7b83748
/home/nolan/others/legacy/bin/sml: Fatal error -- Uncaught exception Fail with "inconsistent state DEFAULT for signal 21" raised at Basis/Implementation/NJ/internal-signals.sml:346.16-346.24

So where are we at this point? We can see that our program has the intended signal handler registered for SIGALRM and it is chosen for handling at least once. Then, with no apparenty declaration of a signal handler signal 21 is chosen. Very strange! A bit more digging is required through the runtime and signal handling, the entirey of ChooseSignal is simple enough:

  /* ChooseSignal:
   *
   * Choose which signal to pass to the ML handler and setup the ML state
   * vector accordingly.
   * WARNING: This should be called with signals masked to avoid race
   * conditions.
   */
  void ChooseSignal (vproc_state_t *vsp)
  {
      int		i, j, delta;

    /* scan the signal counts looking for a signal that needs to be handled. */
      i = vsp->vp_nextPendingSig;
      j = 0;
      do {
	  ASSERT (j++ < NUM_SIGS);
	  i++;
	  if (i == SIGMAP_SZ) i = MIN_SYSTEM_SIG;
	  delta = vsp->vp_sigCounts[i].nReceived - vsp->vp_sigCounts[i].nHandled;
      } while (delta == 0);
      vsp->vp_nextPendingSig = i;

    /* record the signal and count */
      vsp->vp_sigCode = i;
      vsp->vp_sigCount = delta;
      vsp->vp_sigCounts[i].nHandled += delta;
      vsp->vp_totalSigCount.nHandled += delta;

  #ifdef SIGNAL_DEBUG
  SayDebug ("ChooseSignal: sig = %d, count = %d\n",
  vsp->vp_sigCode, vsp->vp_sigCount);
  #endif

  } /* end of ChooseSignal */

Happily, it looks like it still matches the paper pretty well with the comment about masked signals being required to avoid race conditions. That mask is presumably the vp_limitPtrMask we see above in the SML comment. In order for the log line ChooseSignal: sig = 21 to be produced it seems we need to have produced a delta value of 1 which is driven from the state of the vp_sigCounts table. Firing up gdb to look at the runtime code we can see the layout of the vproc_state struct:

(gdb) ptype /o struct vproc_state
/* offset      |    size */  type = struct vproc_state {
/*      0      |       4 */    heap_t *vp_heap;
/*      4      |       4 */    ml_state_t *vp_state;
/*      8      |       4 */    Word_t vp_inMLFlag;
/*     12      |       4 */    Word_t vp_handlerPending;
/*     16      |       4 */    Word_t vp_inSigHandler;
/*     20      |       8 */    sig_count_t vp_totalSigCount;
/*     28      |     248 */    sig_count_t vp_sigCounts[31];
/*    276      |       4 */    int vp_sigCode;
/*    280      |       4 */    int vp_sigCount;
/*    284      |       4 */    int vp_nextPendingSig;
/*    288      |       4 */    int vp_gcSigState;
/*    292      |       4 */    int vp_gcSigThreshold;
/*    296      |       4 */    Time_t *vp_gcTime0;
/*    300      |       4 */    Time_t *vp_gcTime;
/*    304      |       4 */    Addr_t vp_limitPtrMask;

                               /* total size (bytes):  308 */
                             }

And the vp_sigCounts array is made up of the struct:

typedef struct {		/* counters for pending signals; we keep two counters */
				/* to avoid race conditions */
    Word_t		nReceived;  /* the count of how many signals of this variety */
				    /* have been received. This counter is incremented */
				    /* the signal handler */
    Word_t		nHandled;   /* the count of how many of this kind of */
				    /* signal have been handled.  This counter */
				    /* is incremented by the main thread. */
} sig_count_t;

The comments are really quite good and I found the nHandled remark that "this counter is incremented by the main thread" especially interesting given the apparent mismatch in the signals received and counted. If nReceived is incremented by a signal handler and we didn't receive a signal then it stands to reason the handling code is confused somewhere instead.

Stepping through gdb and trying to instrument the SML runtime proved to be challenging. I'll gloss over some of the gory details and mention only that the thing to attach gdb to is the runtime and attaching watchpoints must be done before the SML code begin running. The trick is to break on "RunML" to set a watchpoint after the struct exists but before it is used. An abbreviated gdb session showing as much looks like this:

$ gdb -q .../objs/run.x86-linux
Reading symbols from .../objs/run.x86-linux...
(gdb) break RunML
Breakpoint 1 at 0x804a72b: file ../kernel/run-ml.c, line 97.
(gdb) run @SMLcmdname=sml @SMLload=/home/nolan/others/legacy/bin/.heap/sml.x86-linux repro.sml
Starting program: .../objs/run.x86-linux @SMLcmdname=sml @SMLload=/home/nolan/others/legacy/bin/.heap/sml.x86-linux repro.sml
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, RunML (msp=0x8078780) at ../kernel/run-ml.c:97
97          vproc_state_t *vsp = msp->ml_vproc;
(gdb) watch -l VProc[0]->vp_sigCounts[21].nReceived
Hardware watchpoint 2: -location VProc[0]->vp_sigCounts[21].nReceived
(gdb) watch -l VProc[0]->vp_sigCounts[21].nHandled
Hardware watchpoint 3: -location VProc[0]->vp_sigCounts[21].nHandled
(gdb) continue
Continuing.
Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
[opening repro.sml]
[autoloading]
[library $smlnj/cm/cm.cm is stable]
[library $smlnj/internal/cm-sig-lib.cm is stable]
[library $/pgraph.cm is stable]
[library $smlnj/internal/srcpath-lib.cm is stable]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[library $c/c.cm is stable]
[library $c/internals/c-int.cm is stable]
[library $c/memory/memory.cm is stable]
[New bindings added.]
val it = true : bool
[scanning gen/sqrt.cm]
[loading gen/(sqrt.cm):../libm-h.sml]
[loading gen/(sqrt.cm):callop-0.sml]
[loading gen/(sqrt.cm):fptr-rtti-0.sml]
[loading gen/(sqrt.cm):f-sqrt.sml]
$Execute: required privileges are:
  c-int
  primitive
[New bindings added.]
val it = true : bool

Hardware watchpoint 3: -location VProc[0]->vp_sigCounts[21].nHandled

Old value = 0
New value = 4294967295
0xf7bedce7 in ?? ()
(gdb) 

The mostly confirms things up to this point, we see that nHandled was written to without a corresponding write to nReceived. Interestingly the value written is 4294967295 which is, of course, 0xFFFFFFFF. We saw earlier that is also the way the limit mask works and it is set in preparation for leaving SML code when invoking C code:

           val LimitPtrMask =
               M.LOAD (32, ea (vsp', MS.LimitPtrMaskOffVSP),
                       R.memory)
       in
           (* move vsp to its register *)
           emit (assign (vsp', vsp));
           f { inML = inML, LimitPtrMask = LimitPtrMask }
       end (* withVSP *)

       (* prepare for leaving ML *)
       val () =
       withVSP (fn { inML, LimitPtrMask } =>
                   ((* set vp_limitPtrMask to ~1 *)
                    emit (assign (LimitPtrMask, LW 0wxffffffff));
                    (* set vp_inML to 0 *)
                    emit (assign (inML, LW 0w0))));

Or at least, it is supposed to be assigned to LimitPtrMask. We saw that the vp_limitPtrMask in the vproc_state struct is at offset 304. To try wrapping up this marathon debug session let's ponder - in the above SML we see 0xFFFFFFFF assigned to the name LimitPtrMask which in the object file is at 304 - where is that name or location in the SML code? x86spec.sml:

structure X86Spec : MACH_SPEC =
  struct

    structure DMS = DefaultMachSpecFn (
      struct
	val wordByteWidth = 4
	val addressByteWidth = 4
      end)
    open DMS

    val architecture = "x86"
    val bigEndian = false
    val addressWidth = 32
    val spillAreaSz = 8192
    val initialSpillOffset = 512
    val numRegs = 12 (* 18 *) (* can be |X86CpsRegs.availR| + |vregs| *)
    val numFloatRegs = 21   (* can be |8 upto 31| *)
    val numFloatCalleeSaves = 0
    val startgcOffset = 32
    val pseudoRegOffset = 136
    val constBaseRegOffset = 0
    val fixedArgPassing = false

    val ML_STATE_OFFSET = 176
    val VProcOffMSP = 4
    val InMLOffVSP = 8
    val LimitPtrMaskOffVSP = 200

  end

200 is not where the LimitPtrMask is offset in the vproc_state. What is offset 200 into the vproc_state struct?

The marker for the "handled" signal 21 for which there is no corresponding received marker.

A lot of build up, a bit of a let down

So it seems that at least the 32 bit x86 backend has a misconfigured offset for the limit pointer mask that causes an errant write into the signal handling array at the point the runtime exits the ML code to invoke a C function. I had to go searching around for a sufficiently maintained history of the project because it seems when the repository was moved from subversion(?) to git the history was collapsed. I found this repository which claims to be an archive of the svn history, in it the commit that adds the offset 200 is approximately 25 years old. It is my impression the code worked as expected at that time but when a "new scheme for managing signals" was added approximately 21 years ago it failed to update the offset despite changing the layout of the struct.

A fix

Trying to fix this proves a little interesting because of the nature of the compiler which is bootstrapped and so you'd have to rebuild the compiler and that started to sound a little complicated (or at least tedious). A much simpler (dumber) fix could be to just ensure the vp_limitPtrMask does land 200 into the vproc_state struct. The sigCounts array is too big and is the thing that pushed it out so we'd need to move it but then if we look at the table above we'd be short - so let's just put some padding in:

diff --git a/base/runtime/include/vproc-state.h b/base/runtime/include/vproc-state.h
index 218eb02..d9b2fc0 100644
--- a/base/runtime/include/vproc-state.h
+++ b/base/runtime/include/vproc-state.h
@@ -45,7 +45,6 @@ struct vproc_state {
     Word_t	vp_handlerPending;	/* Is there a signal handler pending? */
     Word_t	vp_inSigHandler;	/* Is an ML signal handler active? */
     sig_count_t	vp_totalSigCount;	/* summary count for all signals */
-    sig_count_t	vp_sigCounts[SIGMAP_SZ]; /* counts of signals. */
     int		vp_sigCode;		/* the code and count of the next */
     int		vp_sigCount;		/* signal to handle. */
     int		vp_nextPendingSig;	/* the index in sigCounts of the next */
@@ -56,7 +55,11 @@ struct vproc_state {
     Time_t	*vp_gcTime0;	    /* The cumulative CPU time at the start of */
 				    /* the last GC (see kernel/timers.c). */
     Time_t	*vp_gcTime;	    /* The cumulative GC time. */
+    char	vp_pad[144];	    /* puts vp_limitPtrMask at byte 200, */
+				    /* where the compiler expects it */
+				    /* (LimitPtrMaskOffVSP, x86spec.sml) */
     Addr_t	vp_limitPtrMask;   /* for raw-C-call interface */
+    sig_count_t	vp_sigCounts[SIGMAP_SZ]; /* counts of signals. */
 #ifdef MP_SUPPORT
     mp_pid_t	vp_mpSelf;	    /* the owning process's ID */
     vproc_status_t vp_mpState;	    /* proc state (see ml-mp.h) */

Compiling the above isn't too difficult but I do get a weird error when invoking the x86-linux makefile for the runtime, though it doesn't seem to matter as the compilation finishes and I get the same error without my patch. Running things with my patched runtime looks like this:

$ sml \
  @SMLrun=$HOME/others/legacy/base/runtime/objs/run.x86-linux \
  repro.sml
Standard ML of New Jersey [Version 110.99.9; 32-bit; November 4, 2025]
[opening repro.sml]
[autoloading]
[library $smlnj/cm/cm.cm is stable]
[library $smlnj/internal/cm-sig-lib.cm is stable]
[library $/pgraph.cm is stable]
[library $smlnj/internal/srcpath-lib.cm is stable]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
[library $c/c.cm is stable]
[library $c/internals/c-int.cm is stable]
[library $c/memory/memory.cm is stable]
[New bindings added.]
val it = true : bool
[scanning gen/sqrt.cm]
[loading gen/(sqrt.cm):../libm-h.sml]
[loading gen/(sqrt.cm):callop-0.sml]
[loading gen/(sqrt.cm):fptr-rtti-0.sml]
[loading gen/(sqrt.cm):f-sqrt.sml]
$Execute: required privileges are:
  c-int
  primitive
[New bindings added.]
val it = true : bool
val root = 1.41421356237 : real
[autoloading]
[autoloading done]
sqrt 2.0 = 1.41421356237
[autoloading]
[autoloading done]
val stop = TIME {usec=1791753912811639} : Time.time
val wait = fn : unit -> unit
finished

Which is neat but only because it doesn't do anything interesting any more.

Thoughts

I really like SML as a language. The speed seems good, development is ergonomic, there are some truly next-generation ideas built in. This process though has me convinced that it may not all be enough to overcome the inertia against it after a quarter of a century. On the one hand, I was able to muddle my way through to an answer for a bit of a thorny problem in some FFI; on the other hand I think I've found a bug that has been around for more than twenty years. Even if I was willing to accept that FFI is 32-bit only it is a tough pill to swallow if just about the first thing I tried turned out to have been incidentally broken (albeit in a minor way that I could work around).

I'm not sure where exactly that leaves me. I've got at least one more minor project in the works but it feels a bit like a dead end given the apparent difficulty I'll face trying to integrate it outside of SML. I could switch implementations to something like PolyML or MLton but neither seems to have as robust an implementation of the thing that drew me to SML/NJ - ConcurrentML.