LuaExec

Version 0.84

Table of contents

• 1. About LuaExec

  • 1.1. Overview
  • 1.2. What it does and doesn't
  • 1.3. Implementation details
  • 1.4. Status, outlook
  • 1.5. License
  • 1.6. Authors, links and downloads

• 2. Installation

  • 2.1. Requirements
  • 2.2. Adjusting the build environment
  • 2.3. Building
  • 2.4. Installation
  • 2.5. Documentation system

• 3. Examples

  • 3.1 Tests
  • 3.2 HTTP server

• 4. Quickstart tutorial

Reference

• LuaExec module reference
• Complete class and library reference manual
• Changelog
• COPYRIGHT

# make help

# make all

# sudo make install

# make docs

local exec = require "tek.lib.exec"
local httpd = exec.run(function()
  require "tek.class.httpd":new {
    Listen = arg[1], DocumentRoot = "doc"
  }:run()
end, arg[1] or "localhost:8080")
-- ... do something else ...
httpd:terminate() -- or httpd:join()

# bin/webserver.lua [host:port]

# lua
> exec = require "tek.lib.exec"

> print(exec.getname())
=> main

> exec.sleep(1000)

> exec.run(function() end)
=>

> exec.run(function() print "hello" end)
=> hello

> exec.run(function() while true do end end)

> exec.run(function() while true do end end)

> for i = 1, 4 do exec.run(function() while true do end end) end

>
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
luatask: received abort signal
stack traceback:
        stdin:1: in function <stdin:1>
        [C]: ?
#

local exec = require "tek.lib.exec"
exec.run(function()
  while true do end
end)
=>
luatask: received abort signal
stack traceback:
        bin/tutorial.lua:3: in function <bin/tutorial.lua:2>
        [C]: in ?

On the other hand, if it has finished voluntarily, everything remains silent:

local exec = require "tek.lib.exec"
exec.run(function()
end)
exec.sleep(100)
=>

I have added the last line to drive home that point, because your funny operating system could theoretically decide to postpone execution of the child task just long enough, so that we ended up with an error and a stracktrace nonetheless. (Told in confidence, this is not true, this example wouldn't raise a signal even without sleeping in the parent, but this is an implementation detail.)

Note that although this is the plushy world of Lua, and LuaExec tries its best to hide the nastiness of multithreaded programming from you, trailing behind a bunch of dangling threads is by no means an encouraged programming technique. What is encouraged is that you synchronize your program on the completion of each task. This also allows you to take delivery of return values:

local exec = require "tek.lib.exec"
local c = exec.run(function()
  return "successful", "indeed"
end)
print(c:join())
=>
true    successful      indeed

Next we should ask ourselves what threads are actually good for. Especially in the Lua world this seems to be hotly debated -

In computations, actually, it is difficult to put them to good use, but matters improved a bit in that regard, with projects like SETI@home, mining bitcoins, and so on. There are plenty of CPU cycles to be wasted in fun computations, especially if you run them in a scripting language. Wouldn't it be a pity if we couldn't saturate eight cores from a one-liner in Lua?

Irony aside, threads have always been useful for long-running tasks, regardless of their computational demand, and they are even more useful if they are bound to I/O operations that are precisely not demanding computationally, but stuttering, jerky, and unpredictable, and spend most of their time waiting, but it is impossible to tell for how long, and if the wait will even turn out a result. Also note that from a computer's viewpoint a human could be seen as a very demanding I/O device, making it a good candidate for a thread or two to deal with, although that may not be the most popular take on that matter.

So in I/O, that's where computing leaves academia and hits the hard cobbles of reality: Imagine you were writing a virus scanner with a graphical user interface (you wouldn't want to write such a software, for all the money in the world, I know! This is just an example) - would you start a task each time the scan button is clicked? No, you would start that task when the application starts, put it to sleep, and in it wait for the order to scan, that is what you'd do. Or rather, that's what I'd do, and I may be completely off the track, because I would allow the user to enter the configuration panels and other sections of the program even while such a scan is in progress.

First let's see how we can start and stop such an application worker:

local exec = require "tek.lib.exec"
local c = exec.run(function()
  local exec = require "tek.lib.exec"
  print "worker running"
  print(exec.wait())
  return "worker done"
end)
exec.sleep(1000)
print(c:terminate())
=>
worker running
t
true    worker done

There is a new function to end the child task, terminate, and another new function, wait, that by sheer chance waits until the main program calls terminate on the child. What wait really does, however, is that it waits until it receives any one or more signals from a set of possible signals, and returns the ones received to the caller. The signal t is included in the default set, and it is important enough to be supported by a function on the other end, terminate, that really does nothing but to send the t signal, and then to behave exactly like join. So this would be equivalent to the last line:

c:signal("t")
print(c:join())

There are more signals, but the t signal is the most gentle one, with the predefined meaning "pretty please, could you quit?". There is nothing wrong if you received this signal and did nothing in reaction, and nothing keeps you from assigning it a different meaning.

By now it has probably dawned upon Unix connoisseurs that LuaExec signals have got nil to do with Unix signals, although there are some artifical similarities in their names. The meanings reserved to them are completely different.

Rowdies and bullies among you (who have an exceptional attention span to be still following this tutorial) surely want to see the brutal termination of a task. We can't help but to admit that this needs to be mentioned, so pay attention, we will do this only once, because we wouldn't want to further support your urge to destroy things:

local exec = require "tek.lib.exec"
local c = exec.run(function()
  local exec = require "tek.lib.exec"
  for i = 1, 1000000000 do end
  exec.wait()
end)
exec.sleep(1000)
c:abort()
=>
luatask: received abort signal
stack traceback:
        bin/tutorial.lua:4: in function <bin/tutorial.lua:2>
        [C]: in ?

It doesn't matter what the child task is doing, as long as it is in Lua code or a LuaExec function, it will be interrupted. Here again the abort method is just the short form for sending a signal, in this case a for abort, and then to join. But why would you want to call abort at all? In exchange you are getting an ugly stack trace that you somehow need to hide from your audience whom you wanted to convince of using your program, and this is even done automatically for you when your program ends and if there are still tasks running. We leave it at that it is a rather exceptional thing to do.

Signals are important, even essential, but their number is limited, only four in fact, of which we know two already, and of which there is only one free for the user, t. There could be more signals, user signals 1, 2, 3, SIGFOO, you name them. But we asked ourselves, why even stop there? No, we have kept the number of signals at the bare minimum so that you won't have to ponder needlessly on freaked out meanings that could be assigned to them.

Because, there is a tool that is so much more powerful, based on, but transcending signals. This tool is called messages. Messages can be any data, you can reserve all meaning in the world to them, and with them, you can even do without signals:

local exec = require "tek.lib.exec"
local c = exec.run(function()
  local exec = require "tek.lib.exec"
  print "your orders please!"
  while true do
    local order, sender = exec.waitmsg()
    print(order, sender)
    if order == "quit" then
      break
    end
  end
end)
exec.sleep(1000)
c:sendmsg("scan")
exec.sleep(1000)
c:sendmsg("quit")
c:join()
=>
your orders please!
scan    main
quit    main

You can send any string (even ones containing unprintable data) around in messages, and not only that, you will also know who has sent you a message, in this example the main program, "main".

By receiving the sender together with the message you can write a worker that is completely ignorant of the rest of the world, it just takes the next order, executes it, and sends the result back to the originator. Our first real worker will compute 5 random numbers between 1 and 10:

local exec = require "tek.lib.exec"
local c = exec.run(function()
  local exec = require "tek.lib.exec"
  while true do
    local order, sender = exec.waitmsg()
    if order == "getrandom" then
      exec.sendmsg(sender, math.random(1, 10))
    elseif order == "quit" then
      break
    end
  end
end)
for i = 1, 5 do
  c:sendmsg("getrandom")
  print(exec.waitmsg())
end
c:sendmsg("quit")
c:join()
=>
8       task: 0x7f4e60000af0    m
3       task: 0x7f4e60000af0    m
7       task: 0x7f4e60000af0    m
7       task: 0x7f4e60000af0    m
9       task: 0x7f4e60000af0    m

Some commentary is probably advised. The first column in the output is the returned message body, the second the name of the sender, as we already know from the previous example. Because here the sender is the child task and the child task has no name, LuaExec has given it a name for identification.

The third return value from waitmsg is the signal that caused waitmsg to return, and by this we also get to know the third signal, m for message, which indicates that, you guessed it, the task has received a message. Please note, however, that it is entirely possible that there is no signal that causes waitmsg to return (in which case the third return value will be nil), because returning the signal to the caller also clears it from the task, and there may be more messages arriving in the meantime, in which case waitmsg will return the next message immediately.

To conclude this introductory tutorial, didn't we say eight cores?

Well, of course none of these exercises in futility have got anything to do with cores, but here's the same worker again (this time we have given it a name), and ten other tasks requesting random numbers from it:

local exec = require "tek.lib.exec"
local c = exec.run({ taskname = "worker", func = function()
  local exec = require "tek.lib.exec"
  while true do
    local order, sender = exec.waitmsg()
    if order == "getrandom" then
      exec.sendmsg(sender, math.random(1, 10))
    elseif order == "quit" then
      break
    end
  end
end })
for i = 1, 10 do
  exec.run(function()
    local exec = require "tek.lib.exec"
    for i = 1, 10 do
      exec.sendmsg("worker", "getrandom")
      print(exec.waitmsg())
    end
  end)
end
exec.sleep(1000)
c:sendmsg("quit")
c:join()
=>
1       worker  m
10      worker  nil
510     worker          workerm
m
8       worker  m
...

You may find the output a bit scrambed sometimes, like I do on my computer. This is harmless and due to the fact that multiple tasks are writing to standard output at the same time. The exact behaviour may vary between different operating systems, but even if you don't get a scrambled output on your computer, you may still get the point.

Of course we can still do worse than that. To expose this tutorial to ridicule once and for all, in the final example the results will be passed on to the main task, which prints them, so the main task becomes the stdout worker, in a sense:

local exec = require "tek.lib.exec"
local c = exec.run({ taskname = "worker", func = function()
  local exec = require "tek.lib.exec"
  while true do
    local order, sender = exec.waitmsg()
    if order == "getrandom" then
      exec.sendmsg(sender, math.random(1, 10))
    elseif order == "quit" then
      break
    end
  end
end })
for i = 1, 10 do
  exec.run(function()
    local exec = require "tek.lib.exec"
    for i = 1, 10 do
      exec.sendmsg("worker", "getrandom")
      local number = exec.waitmsg()
      exec.sendmsg("*p", number)
    end
  end)
end
for i = 1, 100 do
  print(exec.waitmsg())
end
c:sendmsg("quit")
c:join()

Besides prettier output, here we have another feature of LuaExec, addressing a task by the name "*p", which sends a message to a task's parent task.