More efficient stabs.

	Stabs 'compression': more efficient storage.
	Instead of writing all the addresses for which source line
	information exists as dwords, there is now an array with the
	high 16 bits addresses (currently 0x0000, 0x0001 and 0x0002),
	followed by the same number of arrays containing the low 16
	bit addresses.

	Example for current kernel:

		.long 0x40000000 | source_data_count
		.word 3	# nr of high-16 bit addresses
		.word 0, 1, 2	# the high 16 bits of the addresses

		.word 0x2ca8	# 11432 addresses in 0x0000????
		.rept 0x2ca8; .word ?; .endr

		.word 0x3741	# 14145 addresses in 0x0001????
		.rept 0x3741; .word ?; .endr

		.word 0x0a6c	# 2668 addresses in 0x0001????
		.rept 0x0a6c; .word ?; .endr

		.rept source_data_count
			.long sfile_idx << 16 | source_line
		.endr

		.rept nr_source_files
			.long ?	# source file offset relative to sfstr
		.endr

		sfstr:
		.space ?	# asciz source filename string table.

Makefile:
	pass -C to util/stabs.pl

kernel/debugger.s:
	implement finding source line info for compressed stabs table.

util/stabs.pl:
	implement writing and parsing compressed stabs data.
libc: make flat accessible

kernel/libc.s:
	remove setregs/restoreregs, as the selector bases for
	kernel and executables are base aligned.
Relocate symbol table.

bootloader/sector1.s:
	also relocate the addresses in the symbol table.

kernel/debugger.s:
	keep relocation adjustment for source line tables, since
	they are not relocated (yet),
	but don't adjust relocation for symbol table lookups.
PE/COFF: print fix

kernel/pe.s:
	add debug for more calls;
	print the start address again at the end.
Exception handler: verify stack selector

kernel/idt.s:
	when printing user stack, on stack corruption the stack
	selector may be wrong: check for read access (prevents
	exceptions in the exception handler)
Fix tcp memory leak.

kernel/net/arp.s:
	fix errorlabel jump after NET_BUFFER_GET (9f instead of 6f).

kernel/net/net.s:
	after all queued packets are handled, call net_tcp_cleanup.

kernel/net/tcp.s:
	add net_tcp_cleanup, which checks for timed-oud connections,
	frees the buffers, and marks them free. This fixes the memory leak.

	use get_time_ms instead of [clock_ms] as it is fixedpoint 40:24.

	refactor net_tcp_conn_list: extract method to print single
	entry, which can be used for debugging.
Reduce negative time errors.

kernel/pit.s:
	get_time_ms_40_24: move popf to end of method, meaning interrupts
	are disabled, reducing the amount of negative time errors.
	Perhaps a mutex is needed.
[MAJOR] Split kernel into multiple objects.

	SUMMARY

		The build system has been revised to incorporate 'ld'.
		A new source, 'ring2.s', is added, including most of the
		IO drivers, compiled separately.

		The realmode part of the kernel is placed in it's own
		segments .text16 and .data16, which are prepended to .text
		during linking.

		The kernel can now be made in 2 variants: single-object
		and multiple object, yet both will go through the link
		process, yielding functionally identical objects.
		The majority of the work was the linking process,
		including the parsing of relocation tables for merged
		objects, which is not consolidated by ld.

		Effectively there is now one more AS invocation, and two more
		LD invocations:
		* one with -r (--relocatable) to get all the relocation info,
		  yet which does not link global symbols,

		* one without, to link global symbols,
		  yet which lacks any relocation info.
		  (--emit-relocs produces a COFF .reloc section which is also
		   supported in util/reloc.pl, yet the format is badly
		   documented - objcopy -j .reloc produces a file where
		   the documented relocation table begins at offset 0x1000).
		util/checksymbols.pl is no longer called since it was made
		to not call ld (without -r).

		Major work (cleanup aswell) has been done in relocation
		detection code.

	TEST

		Both single-object and multiple object operate correctly
		with and without relocation, which the kernel detects.

	TODO

		It is no longer possible to simply KAPI_DECLARE any method
		anywhere, without also updating ring2.s which holds the
		_KAPI_INDEX from which counting should proceed.

	NOTES:

		'add DEFINE' below means that the file only defines
		content (data, text) if DEFINE=1. It can be included
		with DEFINE=0 aswell, where only constants and macros
		are declared.

		'declare/define once' is similar: it only declares/defines
		if some constant is undefined, and sets it to mark this.
		This was done for those files that are included twice
		due to the multiple object file approach (where the files
		can still be included).

.tcshrc:
	update dk (disassemble kernel) alias: use kernel.obj.
	update dk16 alias: use -j .text16 kernel.o.

Makefile:
	add KERNEL_OBJ = kernel.obj
	add -C to stabs

	FIXME: disable -C -R in reloc.pl call, due to unimplemented bit width
	(likely a negative delta).

TODO:
	add some todo's.

bootloader/sector1.s:
	RELOC_DEBUG = 0
	KERNEL_RELOCATION = 0
	DEBUG_BOOTLOADER override, print some flat values

	reloc32_setup$: support alpha=8, delta=8 or 16

kernel/Makefile:
	link kernel.obj using custom link script

	add configuration: KERNEL_SPLIT_RINGS=1 (separate obj files).

	add AS_DEFS to pass compilation constants as symbols.

	# document some used AS and LD flags.

	no longer call 'checksymbols', since ld is invoked which
	checks the symbols.

	make kernel.bin from kernel.obj, as kernel.o is no longer
	guaranteed to contain all code. Use '-j .text' to only
	copy the .text section.

	make kernel.obj from kernel.o (and possibly ring2.o),
	using a custom linker script (kernel.link), merging
	all sections properly into one big .text section.

	extract $(USB_SRC).

	add targets for ring[0123].s

kernel/kernel.link: (ADD)
	ld link script with comments and disabled parts.
	Defines the labels (in caps) that kernel.s used to define,
	but which now exceed it's scope.

kernel/ata.s:
	add .if DEFINE.

	fix "0: .if ..." -> silent assembler failure, parse errors.

	add 'offset' keyword to IRQ_BASE to force non-mem-ref for
	external constant.

kernel/debug.s:
	move ASSERT_ARRAY_IDX and debug_assert_array_idx here.

kernel/debugger.s:
	move ASSERT_ARRAY_IDX and debug_assert_array_idx to debug.s
	add missing label.
	use [reloc$] instead of .text (since .text is not always
	the kernel start).

	add simple trace debugger: steps through instructions (enter),
	runs until return from current call-depth ('r'), 't' toggles
	trace mode. Can check for specific memory address modification on each
	instruction execution.

kernel/defines.s:
	add .ifndef __DEFINES_INCLUDED - to define once.
	move macro INCLUDE here (from kernel.s)
	move IRQ_SHARING config constant here.

kernel/gdt.s:
	declare once.
	add DEFINE.

	GDT_kernel(Call|Mode|Gate) DEFCALLGATE: use null address,
	set manually due to possible relocation.

	remove macro GDT_STORE_LIMIT, use GDT_SET_LIMIT (works 16/32 bit).

	add macro GDT_PRINT_ENTRY_16 for 16bit debugging.

	init_gdt_16:
		do not calc / set: reloc, realsegflat, codebase, database; this
		is done in realmode.s.

		use [codebase] as base for code selectors.
		use [database] as base for data selectors.
		(both will be the same; if absolutely relocated, they'll be 0;
		 if unrelocated, they'll be [kernelbase];
		 if relatively relocated, it'll be the difference).

		handle nonrelocation for GDTR/IDTR pointers (add [database]
		to [pm_(gi)dtr+1]).

		move stack calculation code to realmode.s.

		remove setup ES, GS, FS

		set kernel_callgate(|_2|_3) using relocated offset.

kernel/idt.s:
	move IRQ_SHARING to defines.s.
	define IDT using 0 address; manual setup in init_idt.

	set pm_idtr+2] using [database] (again - also in gdt.s)

kernel/kapi.s:
	use sections .kapi$(idx|ptr|str|arg), merged in kernel.link.

	don't declare KAPI_NUM_METHODS - defined in kernel.link.

	only declare _KAPI_COUNTER if not defined.

		XXX kludge: ring2.s defines it to be the successor
		    of the nr of KAPI methods declared in kernel.s.

	use 'offset KAPI_NUM_METHODS' to guarantee constant.
	use _KAPI_IDX_SIZE, defined in kernel.link.

kernel/kapi_int80.s:
kernel/kapi_page_callgate.s:
kernel/kapi_page_int.s:
kernel/kapi_page_task.s:
kernel/kapi_sysenter.s:
	use 'offset KAPI_NUM_METHODS' to guarantee constant.

kernel/kernel.s:
	set KERNEL_SPLIT_RINGS from compiler arg __KERNEL_SPLIT_RINGS.
	This flag indicates whether to produce multiple object files
	and merge them through linking, or whether to include all source
	into one assembly step.

	when splitting, IO device code (dev, pci, ata, nics, vid, usb,
	southbridge, virtualisation, sound) is compiled (included) in ring2.s.
	The kernel also then defines global symbols, those that are used
	by the drivers.
	the ring2 code exports the symbols used/referenced by the kernel.

	remove labels, either unused, or replaced by constants defined
	in kernel.link.

	remove signature: define it in linker.s, which knows the kernel end.

	remove kapi data subsection declarations and labels.

kernel/macros.s:
	include once.

	add compiler constant REALMODE_SEP:
		0 means 16 bit code/data is put at the start of .text;
		1 means that separate .text16 and .data16 sectoins are
		used - which means some sort of linking needs to be done.

kernel/mem.s:
	use constants from kernel.link for memory map (mem -k).

kernel/net/httpd.s:
	expression evaluation: use methods to get kernel (code/data)
	sizes instead of label arithmetic, as these labels are
	declared in kernel.link.

kernel/print.s:
	move screen buffer init code from kernel.s to screen_buf_init,
	called from pmode.s.

kernel/pmode.s:
	add [kernelbase], [codebase].
	remove [kernel_location] declaration.

	remove [reloc$], [realsegflat] initialisation.
	remove some legacy [bkp_pm_mode] checks.

	add debug: entry point opcodes, selector base/limit,
	kernelbase, realsegflat, esp.

	clear fs, gs before switching to pmode (no more realmodeFS/GS).

	fix some .code16 which should be .text16 (real_mode_rm).

	call screen_buf_init.

kernel/realmode.s:
	store strings together in .data16,1.

	add macro GETFLAT: reads dword at flat mem edx, returns in edx.

	realmode_kernel_entry: clear one screen, not two.

	set up base pointers:
		[reloc$]: the address to which the kernel is relocated.
		[kernelbase] (cs<<4) the address at which the kernel is loaded.
		[codebase]: kernelbase-[reloc$].
		[databasebase]: (ds<<4)-[reloc$].
		[realsegflat]: 0.

	  Note:
				unrelocated	relocated
	  	[reloc$]	0		0x13000
		[kernelbase]	0x13000		0x13000
		[codebase]	0x13000		0		(SEL_compatCS)
		[database]	0x13000		0		(SEL_compatDS|SS)
		[realsegflat]	0		0

	gdt.s stack top calculation code put here.
	calculate stack top using [kernel_load_end] (below 1mb).

		NOTE! if the kernel is loaded high (say at 1mb),
		then the tables will follow it, and [ramdisk_load_end]
		must be used!

kernel/schedule.s:
	add DEFINE.

	task_wait_io (KAPI): call schedule_near directly, do not use YIELD
	macro, which does another KAPI call.

	cmd_tasks: add sem spinlock for task_queue.

kernel/shell.s:
	move DEFINE up to before cmdline class declaration.

	print error if task_get_by_pid fails.

	cmd_print_idt: print sel as word.

	cmd_pic: only enter cpl0 if needed.

kernel/tss.s:
	print_tss: subtract [database] from TSS pointer from GDT,
	(which is logical and not selector-relative) instead of
	the GDT base for ds. (effect is the same, but GDT_GET_BASE
	macro is larger/slower than [database]).

kernel/vmware/svga2.s:
	remove '$' from 'gfx_mode$'.

util/reloc.pl:
	refactor parsing the relocation info from `objdump -r` into
	a method.

	add method to parse PE/COFF relocation info (.reloc),
	which is produced by 'objdump -j .reloc IN OUT`,
	from an object file linked with '-g' (--emit-relocs).

	add method getsyminfo to parse output of 'nm' into a symbol table hash.

	when parsing relocation information from objdump, the relocation
	value can be a symbolic expression:  '.data-0x......' (or +)
	(where .data can be any symbol). The 'nm' output is used
	to determine whether the value is A(bsolute), and will be
	ignored.

	some fixes for log2($X) -> log2($X|1).

kernel/dma.s:
kernel/mutex.s:
kernel/hash.s:
	add DEFINE.
kernel/pci.s:
kernel/dev.s:
	add DEFINE.define/declare once.

kernel/pic.s:
	add DEFINE

	rename PIC_(SAVE|RESTORE)_MASK: append _static (since unused).
	add PIC_(SAVE|RESTORE)_MASK macros using stack.

kernel/oo.s:
	add DEFINE; declare once.
	fix vptr method relocation.
Make KAPI_* symbols absolute (nonrelocatable).

kernel/kapi.s:
	use '.offset _KAPI_COUNTER; KAPI_\name:' to make the KAPI numbers
	absolute.
	They will still appear in objdump -r(elocation), but this is
	checked against the output of 'nm' and not included in the relocation
	table if the symbol is absolute.

kernel/kernel.link:
	Make KAPI_NUM_METHODS absolute.
	(apparently an expression using absolutes does not yield
	an absolute result).

kernel/realmode.s:
	remove temporary debug level elevation requiring keypress.
Shell: allow command declaration anywhere

kernel/kernel.link:
	merge and place .shellcmd sections in data.

kernel/shell.s:
	store SHELL_COMMAND structure information in .shellcmd section.
Relocation and section fixes.

util/reloc.pl:
	fix regular expression and some cleaning.

kernel/kernel.link:
	add .classdef* sections and section labels.

kernel/oo.s:
	use .classdef, .classdef$* sections.

kernel/shell.s:
	remove unused data section declarations

kernel/defines.s:
	remove unused data subsection numbers

kernel/kernel.s:
	remove unused section labels.

kernel/mem.s:
	disable printing shell command region - no labels.
Add CloudNet framework.

.tcshrc:
	rework/clean some aliases to control virtual machines.

DOC/DNS2.txt:
	Idea on a truly free name service;
	zone authorities as options with majority defaults;
	contesting zone authority;

	service-oriented load balanced routing bootstrapping.

DOC/EnclosedSource.txt:
	add some ideas on open source software-in-an-egg
	(which requires the execution of the software to gain access
	to the source)
	and tracking distribution of derivatives.

kernel/net/cloudnet.s:
	a daemon consisting of two tasks/threads: main and rx.

	zero-config IP address acquisition: tests availability of
	preconfigured DMZ IP and defaults to DHCP when in use.

	UDP broadcast notification of presence: registration.

	All nodes in the local cluster (the broadcast network)
	record the presence of all nodes coming online.

	Simple protocol: asciz followed by binary data.

	All nodes respond to the originator of broadcasts thereby
	allowing the new node to be aware of all active cluster nodes.

	Add commandline interface, showing the status (active:
	repeatedly send 'hello' messages; passive: don't)
	and registry information - the localnet cluster nodes.

kernel/net/dhcp.s:
	add method 'random', using add, shift and xor.

	generate random XID (DHCP session identifiers).

	Attempt to fix reacting (network configuration)
	to DHCP response packets not targeted to the NIC's MAC.

kernel/net/net.s:
	record cloudnet.

	add alias 'net_print_ipv4' to 'net_print_ip' which will become
	deprecated.

	preserve registers in net_packet_print;

kernel/net/socket.s:
	reword SOCKET_READPEER option to also store the destination
	IP and port in the buffer - the header is now 12 bytes, not 6.

kernel/net/udp.s:
	net_put_eth_ipv4_udp_headers: preserve nic (ebx).

kernel/kernel.s:
	make pic.s [clock] global.
	start cloudnet daemon.

kernel/print.s:
	make printdec(8|16|32) global.
	add printdec16.
Update ICMP/DNS SOCK_READPEER handling.
DHCP: fix mac check

kernel/net/dhcp.s:
	in server-to-client BOOTP response messages,
	verify the ETH destination mac (either bcast or our mac),
	and the DHCP client mac before adopting the information.
Add virtual cluster management tool.

.tcshrc:
	add util/ to path.

kernel/net/cloudnet.s:
	fix printing packets; enable throttling.

util/adm:
	perl script configured for vmware, able to add cloned machines
	to a cluster, start and stop them.

	Example:

		> adm vm add
		> adm vm start 0
???

DOC/CloudNet.txt:
	add minor notes on DMZ ip sharing and host OS communication.

kernel/kernel.s:
	make print_time_ms_40_24 global.

kernel/net/cloudnet.s:
	add cluster node structure and array to keep track of states.
	add peer-to-peer cluster_ping.
	add _print_time$: calculates time using [clock].

kernel/print.s:
	update macro LOAD_TXT: add optional third param receiving
	the length of the string for easy cmps.

util/adm:
	add 'vm stop'.

	_vmcmd: pass along extra arguments after parsing the vm id.
	This allows (for vmware) to do:  'adm vm start 1 nogui'.
[MOVE 1/2] Reorganize source tree.

	NOTE: committed in two stages to make sure that git
	recognizes moved files; internal file references
	(includes) are upated in the next commit.

exe/:
	elf.s libc.s pe.s

fs/:
	fat.s iso9660.s sfs.s

kapi/:
	kapi.s, kapi_*.s

lib/:
	buffer.s hash.s ll.s mem.s mem_handle.s string.s token.s

list.s:
	deleted (empty source).

nic/:
	am79c971.s i8254.s rtl8139.s

pcibridge/:
	i440.s ipiix4.s

sound/:
	es1371.s sb.s

usb/:
	usb.s usb_ohci.s
[MOVE 2/2] Update file references

kernel/Makefile:
	add variables for the kernel subdirectories.

	update dependencies for ring2.s.

kernel/kernel.s:
kernel/kapi/kapi.s:
kernel/lib/mem.s:
kernel/ring2.s:
kernel/shell.s:
	update include paths. (unfortunately, .include "x" is relative
	to cwd instead of the file).

	update included filenames.
Various timing updates.

kernel/mutex.s:
	add timeout arg (default -1=infinity) to macro YIELD_SEM.

	update macro YIELD to take optional timeout arg so that a
	specific amount of time can be given to the scheduler.
	Calls YIELD_SEM with a null semaphore.

kernel/net/arp.s:
	net_arp_resolve_ipv4:
		reduce retries to 5 * .5s.
		set CF on retry exhaustion and print msg.
		recode timing: use YIELD timeout and [clock_ms].

kernel/net/icmp.s:
	don't ping again on arp failure:
		net_ipv4_icmp_send_ping returns CF on either ARP failure
		or NET_BUFFER_GET/SEND failure.
		When this happens, do not ping again.

kernel/net/cloudnet.s:
	use -1 (infinite time) for socket_read in cloudnet_rx.

kernel/net.s:
	net_parse_ip: make errormessage fit on one line.

kernel/net/socket.s:
	socket_buffer_read (called by socket_read etc): pass on the timeout
	value given to YIELD_SEM.

kernel/schedule.s:
	add task_io_sem_timeout field to task structure.

	scheduler_get_task$: allow task_io_sem to be null in order to support
	tasks offering a timeout value. When the semaphore is null or 0,
	the timeout will be honored, thus allowing YIELD_SEM to return
	even if the semaphore is still not available.

	task_wait_io: add second stackarg: the timeout. Adjust the given
	time period to the current clock, unless the period is -1.

util/adm:
	add vm suspend,pause,unpause (pause/unpause unsupported by vmrun -T player).

kernel/pit.s:
	leave clock_ms for easy timeout code.
	NOTE: the max time storable is 49 days, and thus timeout scheduling
	may not work entirely right around that time.

	add pit_test to examine clock tick misses when reading latched values.

	get_time_ms_40_24:
	  add and use clock_ms_fp, a 40:24 fixed point, can store about 8 years.
	  add more debug values to print on reading negative time.
	  add TIME spinlock.
Some acpi scan updates..

still no acpi detectable in qemu/vmware.