Re: [ANN] Universal Lua (beta1) -- A LuaJIT Package Manager

  • From: Stefano <phd.st.p@xxxxxxxxx>
  • To: luajit@xxxxxxxxxxxxx
  • Date: Tue, 12 May 2015 20:02:25 +0100

On 12 May 2015 at 17:28, Cosmin Apreutesei <cosmin.apreutesei@xxxxxxxxx> wrote:

Hi again Stefano,

I can give you some feedback regarding the directory layout. I hope
you'll find it useful.

TL;DR: Loading dll depdendencies from separate directories is hard --
much easier to have them all in one directory: the directory of the
exe. Doing otherwise opens up a can of worms that affects many design
decisions when making a package manager.


The long version (sorry for the length but it's complicated):

...

It is very tempting to choose the modular directory layout: it makes
it much easier to write the package manager, it makes browsing the
tree pleasant, and your filesystem API maps cleanly to your package
manager API (removing a package is `rm -rf <package>`, nice!). But
this also means having to modify the runtime to support this new
layout. And some runtimes are less forgiving then others. Let's see
the check list:

* loading Lua modules: just add a custom loader to package.loaders (check!)
* loading Lua/C modules: same here (check!)
* loading dlls: patch ffi.load (clean) or use explicit relative
paths in all invocations (not so clean) (check!)
* loading dependencies of said dlls (or said Lua/C modules)...
here's where it gets ugly: you could use SetDLLDirectory /
LD_LIBRARY_PATH but 1) you would have to know in advance all the
binary dependencies of all your modules (no package database you
say?); 2) you might exceed the max. allowed length for these path
lists with enough modules; 3) it might get slow.

No need to mess with SetDLLDirectory, LD_LIBRARY_PATH and relatives.
And indeed there is no package database.

Locally __meta.lua reports for each module which C-libraries are
needed (see the lcurl pkg).

These dynamic libraries are loaded (via ffi.load) and anchored before
the module itself is required. As each dynamic library is loaded only
once (the name is cached at OS level for the process) this solves the
problem at hand.
The only issue would be if different versions of the same dll are
needed in the same process, but I have no intention to support this in
the short-term (maybe never).

Yes, these dependencies are entered manually into __meta.lua for each
package, but it's no more work than writing down on which other Lua
modules a module depends.

Please notice that right now such dynamic libraries are embedded into
the packages, but this is going to change (probably in the next
release) to facilitate dependency resolution of such dynamic
libraries.

Stefano

Other related posts: