With libraries and microkernel
As I was writing in the previous article, I had to make a decision whether to go microkernel or monolithic kernel way, and I chose the microkernel. Now it is not only a proposed code, but the platform is actually moved to be based on it.
Second big change I made, and by this decision I lost _some_ of the possibilities that were default to appjet, is the decision to merge the world of applications and libraries.
Of course these two coexisted peacefully before, but there still were /* appjet:server */ and /* appjet:library */ sections in library file. Now, the question was: keep this feature (which on the pros side has its simplicity to add testing code to the lib itself), or go another route, by throwing away all mentions of /* appjet:library */ from the developer.
To understand why this question appeared at all, one must understand how multi-app is (to be) implemented in this platform - an app is started by import("lib-something") and lib-something is built by the platform itself, just to make it possible to run the application code by calling the import.
Mechanistic approach is: for any application foo, let us create, for example, lib-!foo library, and let us prohibit "!" in regular names. That means for application fb-statuses, platform would create lib-!fb-statuses to run its server code section. For application lib-mmos-view, the platform creates lib-!lib-mmos-view to run the server code section, but it also creates lib-mmos-view, to enable importing its library section by other applications. I hope it's not too complicated.
Now the question was: what if we simply forget that applications are of two types - apps and libs, and there were only one type, but it could be utilised in two ways: either you run it, as an application, by writing http://foo.appdomain.net/, or you import it in any other application by calling import("lib-foo")? It looks like the application is there two times: once as an application, and once as a library. But if you forget app/lib dichotomy, there is just one bunch of code, which can work simultaneously as an app or as a lib.
This would greatly simplify the innards. For the app/lib foo there would only be one file - lib-foo. If you want to create an application, you create foo and then run it through url. If you want to create a library, you create bar and then import("lib-bar"). And if you can utilise both approaches, you are free to do it.
So, as everyone already guessed, I decided for this unified, one-file way. That means I cannot write the tests for a lib into its code, but no one prevents me to just create bar-tests application which runs the tests. The elegance of one-file solution is gone, but I finally let it go (partially because I do not plan to use full-file editing, anyway, but edit each code section separately; in that case I lost nothing, since there is no notion of "one file" anymore).
I restructuralized the platform to consist of boot.js (not editable anymore, only libs are), lib-admin which includes all the rescue, create and edit functionality, and the rest (which is now only two very little libs: lib-worker and lib-0). As in previous case, the platform is set up by running the server with configure.js as the main file, in two steps, with first step setting up the storage and the second step rescuing lib-admin. It's similar in this to previous configure, only it sets up a little differently structured beast:
// (c) 2009, Herbert Vojčík
// Licensed by MIT license (http://www.opensource.org/licenses/mit-license.php)
// ====== variables to configure ======
var pwd = "undefined";
var domain = "appdomain.net";
// please configure the path to the save script in saveRawSource below if needed
// ====== verbatim copy of lib-admin functions ====== begin ======
function appDataSafe(app) {
try {
return getStorable("obj-! " + app);
}
catch (ex) {
return null;
}
}
function appData(app) {
var result = appDataSafe(app);
if (result) {
return result;
}
printp(request.path, ": unknown app: ", app);
response.stop();
}
function appDataForce(app) {
var result = appDataSafe(app);
if (result) {
return result;
}
appjet._native.storage_create("obj-! " + app);
result = appData(app);
storage.apps.add(result);
return result;
}
function saveRawSource(files) {
var info = {pwd:storage.pwd};
for (var p in files) {
info[p + ".js"] = files[p];
}
wpost("http://staticdomain.net/fs.rb", info);
}
function saveAppSource(name, code, doBackup) {
var appdata = appData(name);
if (doBackup) {
appdata["server.bak"] = appdata["server"];
}
var files = {};
files["lib-" + name] = "/* appjet:version 0.1 */\n" + "/* appjet:library */\n" + code;
saveRawSource(files);
appdata["server"] = code;
}
function get_main() {
print(OL(LI(link("/rescue")), LI(link("/edit")), LI(link("/create"))));
}
function get_rescue() {
var app = request.params.app;
var appdata = appData(app);
var rescue = appdata["server.bak"];
saveAppSource(app, rescue);
printp("Last known good code of ", app, " restored.", BR(), CODE(rescue));
printp(link("http://admin." + appjet.mainDomain + "/edit?app=" + app, "Go edit it!"));
}
function get_edit() {
var app = request.params.app;
var appdata = appData(app);
page.head.write("<script>\n" + " var c=" + {s:appdata.server} + ".s;\n" + " window.onload=function(){var ta=document.getElementById('code');ta.defaultValue=ta.value=c;};\n" + "<" + "/script>");
page.setTitle(app + html(" – ") + "admin editor");
printp(B(app, ".", appjet.mainDomain), BR(), B({style:"color:#0c0"}, "/* appjet:server */"), BR(), FORM({method:"post", action:request.path}, TEXTAREA({id:"code", name:"code", style:"width:100%;height:80ex"}), INPUT({type:"hidden", name:"app", value:app}), INPUT({type:"submit"}), " ", INPUT({type:"reset", value:"Undo"})));
}
function post_edit() {
var app = request.params.app;
var appdata = appData(app);
var code = request.params.code;
if (code) {
saveAppSource(app, code, true);
}
response.redirect(request.path + "?app=" + encodeURIComponent(app));
}
function get_create() {
page.setTitle("admin app creation");
print(FORM({method:"post", action:request.path}, "Supply application name:", INPUT({name:"app"}), INPUT({type:"submit"})));
}
function post_create() {
var app = request.params.app;
var appdata = appDataSafe(app);
if (appdata) {
printp("Application ", B(CODE(app)), " already exists.");
get_create();
return;
}
if (!app.match(/[a-z][a-z0-9-]+[a-z0-9]/)) {
printp("Application name must", UL(LI("be at least three characters long,"), LI("begin with smallcaps letter,"), LI("end with smallcaps letter or digit,"), LI("and contain only smallcaps letters, digits, or minus sign/dash (-) characters.")));
get_create();
return;
}
appdata = appDataForce(app);
appdata["server.bak"] = "printp(\"Hello, world!\");";
response.redirect("http://admin." + appjet.mainDomain + "/rescue?app=" + app);
}
// ====== verbatim copy of lib-admin functions ====== end ======
switch (request.path) {
case '/1':
var adminBuilder = [
"""// (c) 2009, Herbert Vojčík
// Licensed by MIT license (http://www.opensource.org/licenses/mit-license.php)
"""
];
[appDataSafe, appData, appDataForce, saveRawSource, saveAppSource, get_main, get_rescue, get_edit, post_edit, get_create, post_create, ].
forEach(function(f) { adminBuilder.push(f); });
adminBuilder.push("""
import("storage");
appjet._internal.global = this; //dispatch fix
dispatch();""");
// ====== Storage dump, edited ====== begin ======
(function () {
var _an = appjet._native;
_an.storage_create("obj-! 0");
_an.storage_create("obj-M1OtYQoF");
_an.storage_create("obj-! admin");
_an.storage_create("obj-! worker");
_an.storage_coll_create("coll-LyqehQFl7");
_an.storage_put("obj-! 0", "server.bak", "string",
"""// (c) 2009, Herbert Vojčík
// Licensed by MIT license (http://www.opensource.org/licenses/mit-license.php)
if (request.headers.Host === "admin."+appjet.mainDomain) {
appjet.appName = "admin";
appjet._internal.queue.unshift("lib-admin");
} else {
import("lib-worker");
}""");
_an.storage_put("obj-root", "pwd", "string", pwd);
_an.storage_put("obj-root", "bootTable", "object", "obj-M1OtYQoF");
_an.storage_put("obj-root", "apps", "object", "coll-LyqehQFl7");
_an.storage_put("obj-M1OtYQoF", "admin."+domain, "string", "admin,,lib-admin");
_an.storage_put("obj-! admin", "server.bak", "string", adminBuilder.join(""));
_an.storage_put("obj-! worker", "server.bak", "string", "printp(\"Hello, world!\");");
_an.storage_coll_add("coll-LyqehQFl7", "obj-! worker");
_an.storage_coll_add("coll-LyqehQFl7", "obj-! admin");
_an.storage_coll_add("coll-LyqehQFl7", "obj-! 0");
})();
// ====== Storage dump, edited ====== end ======
printp("Storage configured. Backup version of admin, 0 and worker apps included; you may restore them afterwards using admin."+domain+"/rescue.");
printp("MD5 of password is:", BR(), md5(pwd), BR(),
"Please update the hash in the filesaving script and proceed to ", link("/2", "step 2"), " to restore lib-admin.js.");
break;
case '/2':
import("storage");
request.params.app="admin";
get_rescue();
printp("admin.js restored.");
printp("You may stop running configure.js. ",
"As the next step, set up your domain ("+domain+") in the last line of boot.js. ",
"Then start appjet with boot.js as the main file.");
break;
default:
printp(H1("Platform configuration"));
printp(BIG(link("/1", "Start the wizard")));
break;
}Moving from old boot.js to microkernel and lib-admin enabled one thing - the cumbersome use of appjet._native.storage_xxx is no longer needed and lib-admin (being an app, a special one, but an app) can simply import("storage") (the reason why appjet._boot in previous boot.js could not import("storage") as well lies in the security, to be able to virtualize storage it must not be imported and used by the platform itself, but the user must be the one who imports it as the first - otherwise storage.js caching permits access to unwanted parts of the storage. I'll write more of this when I actually do the storage virtualization).
Second thing, which I already mentioned in "Microkernel" post, is that normal apps are run straight, without loading and compiling admin parts, like the filesaving. It is needed only by lib-admin, so only there it is.
Of course, fs.rb is also part of the distribution, but it was not changed at all, so I do not copy it again. Last, but not least, the configure.js does _not_ write down boot.js (it writes down lib-admin), so boot.js is also part of the distribution, as the third file:
/* appjet:version 0.1 */
// (c) 2009, Herbert Vojčík
// Licensed by MIT license (http://www.opensource.org/licenses/mit-license.php)
(function(amd) {
var _an = appjet._native;
var tableid = _an.storage_get("obj-root", "bootTable").value;
var appBoot = _an.storage_get(tableid, request.headers.Host).value;
var name, queue = appjet._internal.queue = appBoot ? appBoot.split(",") : ["0", "", "lib-0"];
appjet.appName = queue.shift();
appjet.mainDomain = queue.shift() || amd;
while((queue = appjet._internal.queue) && (name = queue.shift())) {
var agent = import({}, name);
if (typeof agent.runAgent === "function") { agent.runAgent(queue); }
}
})("appdomain.net");You must manually setup your domain in boot.js, as configure.js tells you. And that's it. (Oh yes, and you should rescue the 0 and the worker). The setup process is not as nice as previous one - you must configure your domain in two places. I think I'll fix it for the next phase, and configure.js will write boot.js down as well.
As you may have noticed, the most precious part of the platform is now lib-admin, much like boot.js was before. It contains all the rescue, create and edit functionality (as yet). That's why there is a "safety belt" in lib-0 (as you can see from its code in configure.js) - if the standard, straight running of admin.appdomain.net url stops working for some reason, and the fallback lib-0 is imported instead, it tests for admin specifically and schedules it to run anyway, if the url matches.
What I plan to do next: first, get rid of the "obj-! "+name constructed storage ids, they are dangerous to use. There is a bug in storage implementation, and if you recreate existing id, it disappears after some random time, and keeps disappearing if created again. You must somehow export the storage, delete it, let the appjet create new one and import data back.
Next, the fixes to configure to be able to setup domain ar one place only.
Then there are more challenges, but I will probably set as a medium sized goal the ability to actually make the platform safely public, so multiapp, some kind of security and a little better editor (as a standalone app, not in lib-admin, it's an "emergency" editor, in fact, there must be one, but I don't plan to develop it further, it does its work decently).
ps. Yes, and I hope to create solid base soon (by solid base I mean such a "distribution" (that is, configure.js), which will not have to be changed any more, from which working base of a platform can be set up; and any future version of the platform could be built form this base by just adding new applications, changing existing ones (admin, 0, worker) or using admin consoie. In other words, so every future version can be defined just as The Base + bunch of sourcecode to add to it via editor(s) / to run in admin console). Even this one looked promising, but not yet. The next one, hopefully.
pps. Yes, and one more little thing. :-/ I will need to have to change the filesaving service. I realized I not only need to write files, but also to delete them, sometimes... I will do it simply - if I want to delete the file, I just ask the filesaving service to write nothing to it (truncate it to empty file), and use this as a protocol for "please, delete the file").