2011-07-16
How Cool Is Tokyo?
V850ES/Jx3-L Starter Kit features a V850ES/JG3-L MCU (μPD70F3793).
It's using a 78K0 MCU (μPD78F0730) as USB UART bridge.
Here's a USB trace excerpt:
03 01
01 03
[IN w/0x200, no reply]
00 80 25 00 00 01
04 00 00
01 02
01 00
00 80 25 00 00 01
04 00 00
02 fa cf
04 00 00
00 80 25 00 00 01
04 00 00
00 80 25 00 00 01
04 00 00
01 00
00 80 25 00 00 01
04 00 00
02 fd ff
04 00 00
00 80 25 00 00 01
04 00 00
00 80 25 00 00 01
04 00 00
01 00
01 00
00 80 25 00 00 01
04 00 00
02 00 00
04 00 00
00 80 25 00 00 01
04 00 00
00 80 25 00 00 01
04 00 00
01 01
01 01
00 80 25 00 00 01
04 00 00
02 01 00
04 00 00
00 80 25 00 00 01
04 00 00
2011-07-04
Learning to SWIM
Apparently there is no free USB sniffer for Windows 7;
so on Saturday I dug out and replugged a physical Windows XP machine
(my Planche SVN server btw).
SniffUSB 2.0
did the job okay under Windows XP.
Reading the flash from ST Visual Programmer resulted in a 7.2 MB log file.
On Sunday I was done reading the firmware of my STM8S-Discovery on Darwin!
While the ST-Link is my first USB project, I did write an iSCSI client for the Haiku bootloader,
so the CDBs were somewhat familiar. Not the vendor-specific ones of course!
Why all the fuss, you might wonder. And yes, there's a User Manual on SWIM.
But it tells you how long to set the line to low or high etc.
I couldn't find a direct correlation between the three SWIM commands (SRST, ROTF, WOTF)
and the CDBs.
Exiting SWIM mode was trivial once reading worked, and it filled a gap in the enum.
While deciphering the 3.5 MB flash write mechanism,
I found that the buffer is not cleared between commands,
so may contain artefacts at unused offsets.
2011-06-28
Libusb breakthrough for ST-Link
Sunday I've hacked on a libusb-based program to interface with my STM8S-Discovery's ST-Link.
The device's USB interface couldn't be claimed under Darwin though, since Darwin's
IOUSBMassStorageClass driver tried to use it apparently. The equivalent to a Linux "quirk"
appeared to be a so-called codeless kext, which assigns a device/interface a different driver.
But all variations that I tried didn't work out. I even tried writing a real stub kext.
Today I found out that the stupid property list editor had been using a wrong value type
for some of the match criteria. And suddenly it works!
It turns out that STM8S-Discovery, STM8L-Discovery and STM32VLDiscovery all have the same
product and vendor IDs, start in "DFU" mode and return ST-Link version 1.
STM32VLDiscovery has JTAG version 11 and no SWIM version.
Both STM8S-Discovery and STM8L-Discovery have SWIM version 3 and no JTAG version.
Exiting DFU mode (to mass storage mode) works okay and survives multiple program invocations.
2011-06-23
Sunshine fading
It's been a while that Oracle has taken over Sun and I still hadn't made the move away from
discontinued OpenSolaris. So I tried upgrading from snv_134b to today's
OpenIndiana -
a nightmare!!! I've managed to uninstall (cf. below) the pkg command
and haven't figured out a way to get it back, so I'm pretty much destined to
install OpenIndiana from scratch. Lots of data to backup from ZFS first,
including QEMU VMs and my interrupted PowerPC port of Haiku.
$ pfexec pkg install SUNWipkg SUNWipkg-um SUNWipkg-gui
$ pfexec pkg uninstall entire slim_install system/zones/brand/ipkg
$ pfexec pkg uninstall package/pkg # DON'T!!!
$ pfexec pkg image-update -v
2011-06-09
After POWERing up
To boot from CD I found this to work:
0 > boot /pci@800000020000003/pci@2,3/ide@1/disk@0
2011-06-02
POWERing up again my IBM IntelliStation 9111-285
screen how-to
It's been a while, and not only do I have to look up each time how to connect to the
ASMI (Advanced System Management interface) but also how to connect to the serial port from my PowerMac:
$ screen /dev/tty.usbserial 19200
and how to get out the proper way (Ctrl+A,Ctrl+K,y works for me).
OF device names
0 > devalias
ibm,sp /vdevice/IBM,sp@4000
disk /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0/sd@8,0
network /pci@800000020000004/pci@2,4/ethernet@1
net /pci@800000020000004/pci@2,4/ethernet@1
network1 /pci@800000020000004/pci@2,4/ethernet@1,1
scsi /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0
nvram /vdevice/nvram@4002
rtc /vdevice/rtc@4001
screen /vdevice/vty@30000000
ok
OF settings
0 > printenv
-------------- Partition: of-config -------- Signature: 0x50 ---------------
ibm,fw-dc-select 100 0
ibm,fw-default-mac-address? false false
ibm,fw-forced-boot
ibm,fw-n-bc 255.255.255.255 255.255.255.255
ibm,fw-n-bretry 00 00
ibm,fw-n-tretry 00 00
ibm,fw-n-dbfp 00000000 00000000
ibm,fw-n-dafp 00000000 00000000
ibm,fw-n-rc A A
ibm,fw-n-ru Y Y
-------------- Partition: common -------- Signature: 0x70 ---------------
little-endian? false false
real-mode? true true
auto-boot? true true
diag-switch? false false
fcode-debug? false false
oem-banner? false false
oem-logo? false false
use-nvramrc? true false
ibm,fw-tty-language e 1
ibm,fw-new-mem-def true false
ibm,fw-prev-boot-vpd
ibm,fw-keyboard 1 1
real-base 2000000 2000000
virt-base ffffffff ffffffff
real-size 1000000 1000000
virt-size ffffffff ffffffff
load-base 4000 4000
screen-#columns 64 64
screen-#rows 28 28
selftest-#megs 0 0
boot-device /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0/sd@8:2
boot-file
diag-device /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0/sd@8:2
diag-file diag diag
output-device /vdevice/vty@30000000 com1
input-device /vdevice/vty@30000000 com1
oem-banner
oem-logo
nvramrc Defined : use NVEDIT related words to view
boot-command boot boot
reboot-command
menu? false false
ibm,fw-find-tape-alias false true
ibm,fw-find-cdrom-alias false true
ibm,dasd-spin-interval 5 5
bootinfo-aix /pci@800000020000003/pci@2,4/pci1069,b166@1/scsi@0/sd@8:3
ibm,fw-vkbd-input /pci@800000020000004/pci@2,3/usb@1/hub@1/hub@1/keyboard@1
ibm,fw-00145e9703ee 100,full
ibm,fw-00145e9703ef 100,full
ok
OF device tree
0 > ls
000002087d68: /ibm,serial
00000208aab0: /chosen
00000208ad08: /packages
00000208ae20: /disassembler
000002091210: /assembler
0000020bc458: /dev-tree
0000020bcc38: /lpevents
0000020c4da0: /deblocker
0000020c5e60: /disk-label
0000020ca908: /tape-label
0000020cac10: /obp-tftp
0000020d9920: /ip
0000020dfa90: /prep-boot
0000020e02d0: /fat-files
0000020e2668: /iso-13346-files
0000020ebf08: /utilities
000002105138: /net
00000210ab50: /iso-9660-files
00000210bf28: /boot-mgr
000002120010: /chrp-loader
000002120210: /pe-loader
000002120ae0: /elf-loader
000002124338: /nls-support
0000021250d0: /terminal-emulator
0000021251e8: /dynamic-reconfig
000002215ef8: /gui
0000022318f8: /iscsi
000002248f68: /post
0000020a3a68: /cpus
0000020a7fa8: /PowerPC,POWER5@0
0000020a90a8: /l2-cache@2000
0000020a94c0: /l3-cache@3020
0000020a9c28: /memory@0
0000020b41f8: /ibm,dynamic-reconfiguration-memory
0000020b8f70: /options
0000020b9cd0: /aliases
000002125720: /openprom
000002126dd0: /ibm,bsr@30002000000
0000021273e8: /event-sources
000002128fa0: /epow-events
000002129128: /interrupt-controller@3fe0f000000
00000212a9e0: /interrupt-controller@3fe0000a400
00000212c7e0: /interrupt-controller@3fe0000b400
00000212e5e0: /interrupt-controller@3fe00008400
0000021303e0: /rtas
000002136c20: /vdevice
000002139b78: /vty@30000000
00000213c968: /vty@30000001
00000213fb18: /IBM,sp@4000
000002140bc8: /rtc@4001
0000021414d0: /nvram@4002
000002141bd8: /pci@800000020000002
000002149818: /pci@800000020000003
000002151418: /pci@2,3
000002199668: /ide@1
00000219c2d0: /disk@0
000002157f80: /pci@2,4
0000021a1a50: /pci1069,b166@1
0000021ab790: /scsi@0
0000021ade10: /sd
0000021af850: /st
0000021b13b8: /scsi@1
0000021b3a38: /sd
0000021b5478: /st
00000215eae8: /pci@2,2
000002165798: /pci@2,6
00000216c448: /pci@2
0000021730f8: /pci@800000020000004
00000217acd0: /pci@2,3
0000021b70b8: /usb@1
0000021d35e0: /hub@1
0000021d48d8: /usb@1,1
0000021f0e00: /hub@1
0000021f20f8: /usb@1,2
0000021f2930: /hub@1
000002181870: /pci@2,4
0000021f3350: /ethernet@1
000002202f68: /ethernet@1,1
0000021883f0: /pci@2
00000218f0a0: /pci@2,2
ok
2011-05-22
The dark side of ni(hi)lism
I managed to screw up my nilfs2 partition.
As gildean says, NILFS is "quite different from the classical filesystems".
It indeed is, it doesn't clean up its blocks itself, it needs a daemon to occasionally do so,
installed by nilfs-tools. That's not available
through apt-get, so you need to download the .deb and double-click it.
In order to transfer the on-SD user to MMC, I mounted /dev/mmcblk3p6 to /mnt and ran
cp -R --preserve . /mnt/myuser and set up an entry in /etc/fstab.
2011-05-21
Linux running
Turns out that some good soul prepared an alternative boot.mmc.32.img
file elsewhere.
Linux 2.6.32 runs pretty well now, so unless the bootloader changes again I won't care about
Android any more.
2011-05-19
Froyo FTW ... or better not?
I updated my Toshiba AC100 to 2.2.5.0029
(Android "Froyo")
last week, only to find out that it started booting into my Linux partition.
Lucky me, I had done a backup of the original partition!
With partition 5 restored, the system update completed successfully after some time.
In addition to the annoyingly crappy email client,
there turns out to be no Facebook app on Toshiba Market Place. ;)
Attempting to install Linux again
Here's the steps I tried yesterday:
For size reasons, boot.32.img cannot go to partition 5 as before.
You'd get a strange "failed executing command 14 NvError 0x120000" message trying.
So do a backup of partition 6 first.
sudo LD_LIBRARY_PATH=. ./nvflash --bl ../prebuilt/fastboot.stock.bin --download 6 ../../path/to/boot.32.img --go
This replaces the main kernel+initrd with phh's one.
The Linux image has changed the file system from ext3 to nilfs2, so I needed to install
the nilfs-tools package. Then, Ubuntu's volume manager is able
to format the partition with NILFS; don't worry it won't recognize the format and display it as
"Unknown". So to mount it, look up the device path (here, /dev/sdb1)
and mount it manually (here, at /mnt).
sudo mount -t nilfs2 /dev/sdb1 /mnt
After the usual
sudo tar xavf Ubuntu7.tar.gz --numeric-owner -C /mnt
I had to unmount it manually
sudo umount /mnt
Unfortunately on boot it complains about not finding /dev/mmcblk3p6.
Seems like the SD card is not scanned by the init script?
2011-03-24
ST-Link findings
It seems Martin Capitanio
recently succeeded in accessing the STM32 Value line discovery ST-Link from Linux!
There's a fork on GitHub
that claims to have a working gdb-server.
For starters I'm struggling to get arm-none-eabi GCC 4.5.2 compiled
in order to download the current firmware with
stlink-download.
Andreas Schwab's recipe for ppc64 luckily works here, too: make all-gcc; sudo make install-gcc
2011-03-20
µC tools WIP
Lately I've been spending some time working with microcontrollers again.
At university I had been working with msp430 based sensor nodes
as well as 68k and avr based boards.
My STM8-Discovery had been
collecting dust, with no stm8 binutils / GCC available and no evaluation key from Cosmic arriving.
So, inspired by the chatter about the LLVM
keynote at FOSDEM 2011,
I decided to get to know LLVM and clang by contributing
an STM8 backend.
This Embedded World, the STMicroelectronics people rather pointed me to the
STM32-Discovery of MCUs
than caring about the STM8 compiler situation
(and Atollic do seem to ship a binary
arm-eabi GCC for their Eclipse-based
TrueSTUDIO/STM32).
But they were willing to give me an STM8L-Discovery for comparison.
That was when I realized how gently I had been introduced to the world of microcontrollers
by our university's msp430 Linux setup; and probably why their Embedded Systems lab
was running some oldish Windows.
Diving deeper into the world of MCUs
A nice lady from Silicon Labs
(note: a few years earlier one man simply sent me to silabs.com, as did NXP this year)
gave me not only the HID-USB-to-IR reference design
but also a C8051F336DK development kit,
saying openly that their tools were for Windows only.
But I did find an SDCC project.
Renesas provided me with a V850ES-Jx3-L starter kit
and promised to ship me an RL78 starter kit as well (but then came the earthquake in Japan).
I was really surprised to learn that Renesas these days is also supplier
of SuperH (e.g., sh4) MCUs.
Renesas' tools seem Windows-only,
but supposedly binutils / GCC have support for v850-elf.
Still need to check.
LLVM status
In my LLVM quest I succeeded in compiling trivial C methods to stm8 assembler.
This weekend I managed to prefix the file with the required stm8/ line, based on Subtarget.
But it still emits a lot of ELF-specific cruft that the ST assembler wouldn't understand;
I fear I may need an MCStreamer to teach LLVM to emit assembler source
that ST can process (avoiding .file, .align, etc.).
Either that or implement an MCCodeEmitter that assembles machine code directly -
the advantage would be the ability to do it on Darwin, the downside is that there's no
documented instruction formats wrt src/dst encoding so I would need to encode/define each
instruction variant.
2011-03-20
Tegra 2
For New Year I got myself a Tegra2-based (arm) Toshiba AC100 netbook.
At FOSDEM 2011 I spoke to the speaker of a MeeGo talk about possible
Tegra2 support.
At Embedded World 2011 the Nvidia guys unfortunately were unsensitive to my issues
and pointed to companies like Toradex. Nice people indeed and open to upstreaming things,
but obviously they can't easily lift the restrictions Nvidia puts on us with their binary
x86 Linux tools like nvflash and their L4T's binary userspace daemon(s) and Xorg drivers...
2010-11-01
ppc64 debugging breakthrough
To get powerpc64-linux-gnu-gdb compiled,
I had to remove the $(LIBS) dependency of the psim target.
2010-08-29
No longer scuzzy: iSCSI
Sometimes a little distance helps solving problems. Having worked towards iSCSI boot support for Haiku some time already,
I kept getting an extra SCSI response with status 0x2 when doing READ (6), READ (10) or READ (12).
Turns out it wasn't related to the read command at all. Instead of the SCSI response message to READ CAPACITY
I expected I was getting a Data In message all along, followed by the actual SCSI response message with status 0x0.
Having found out, it turned out that READ (12) was indeed triggering an invalid opcode sense key.
Using READ (16) worked much better!
I committed its struct in r38425,
maybe Adrien or someone has a use for it too.
The boot started to go pretty far then. I ran into memory issues though and thought it might be a bad idea to
malloc ~1.6 MiB for the first kernel_ppc ELF segment, so I implemented a zero-copy mechanism instead.
Getting this right for the case where we don't want full blocks got my mind a little twisted.
Anyway it's working now, I get to the frame buffer (third icon) and see the PIC panic.
Yesterday I briefly poked at ppc64 bootloader support and got the memory ranges detected correctly.
Still unsure how to handle the memory translations.
2010-06-12
Mr. MMU Man #2
Seems I never documented the initial memory mappings on my PowerMac G3, so here we go:
checking for memory...
0: base = 0x00000000, size = 134217728
1: base = 0x08000000, size = 134217728
2: base = 0x10000000, size = 134217728
3: base = 0x18000000, size = 134217728
total physical memory = 512 MB
suggested page table size = 4194304
need new page table, size = 4194304!
new table at: 0x00400000
[...]
found 12 translations
0: map: 0x00000000, length 12288 -> physical: 0x00000000, mode 16
found exception handlers!
1: map: 0x00102000, length 212992 -> physical: 0x00102000, mode 2
2: map: 0x00137000, length 176128 -> physical: 0x00137000, mode 2
3: map: 0x00400000, length 4194304 -> physical: 0x00400000, mode 16
found page table!
4: map: 0x80800000, length 524288 -> physical: 0x80800000, mode 40
5: map: 0x80882000, length 4096 -> physical: 0x80882000, mode 40
6: map: 0x80a00000, length 65536 -> physical: 0x80a00000, mode 40
7: map: 0x90000000, length 268435456 -> physical: 0x90000000, mode 40
8: map: 0xfe000000, length 65536 -> physical: 0xfe000000, mode 40
9: map: 0xfec00000, length 4096 -> physical: 0xfec00000, mode 40
10: map: 0xfee00000, length 4096 -> physical: 0xfee00000, mode 40
11: map: 0xff800000, length 2097152 -> physical: 0x1fe00000, mode 16
[...]
#1 is the boot loader (__text_begin at 0x00102074, size to _end 221068)
#3 is the memory claimed by Haiku for the page table.
#7 contains the ATI Radeon 9200 frame buffer (0x98008000).
#11 is the OpenFirmware (entry at 0xff809b08).
r37001 hangs after:
free boot range: also remove 86d80000 - 0xfe00ffff
Bug in vm_free_boot_loader_ranges
ppc_exception_entry[...]
heap_add_area[...]
INIT: init Device Mapper
PANIC: remove page [...] from cache [...]: page still has mappings
Welcome to Kernel Debugging Land...
[...]
endless loop
2010-05-13
„Seh' ich so hungrig aus?“
Diese Frage stellte mir eine Kollegin, als ich sie beim Mittagessen vorließ.
Auf Fragen zum Aussehen gibt es nicht wirklich eine gute Antwort:
Würde ich "ja" sagen, könnte sie denken, sie sehe dürr aus.
Würde ich "nein" sagen, könnte sie denken, sie sehe dick aus.
Manchmal ist die beste Antwort keine Antwort! Oder wie ein Wettbewerber es formuliert hat:
„Däs isch, wie wenn mei' Frau fragt: Bin ich zu dick?!“
2010-05-13
Dissecting network hardware
I've been having some network hardware issues the week before, so I looked into reflashing
my D-Link access point with OpenWRT.
The power supply unit and possibly the flash is damaged, and soldering the serial port is not yet finished.
Had a look into my old LG router as well,
while at it.
2010-01-14
TCP/IP
Last weekend I've hacked together the beginnings of a TCP implementation for the Haiku boot loader.
It handles the SYN, SYN-ACK, ACK sequence for connecting and FIN-ACKs for disconnecting.
Queuing and resending data packets are prepared but untested.
RemoteDisk uses a UDP-based protocol, which shows a little unstable latency over my PowerMac G3's
built-in Ethernet connection (100 MbE client - 1 GbE switch - 1 GbE server).
Unfortunately my supposedly Mac-compatible GbE PCI network card only works with a driver under Mac OS X
but is not recognized by OpenFirmware as device_type network.
Similarly the built-in Firewire is not recognized as device_type ieee1394,
otherwise IP over Firewire would've been an option to boost network performance by four.
2010-01-06
Haiku ppc
At r34915:
INIT: init VM semaphores
arch_vm_init_end(): 2 virtual ranges to keep:
start: 0x00131000, size: 0x9000
no kernel address, skipping...
start: 0xff800000, size: 0x200000
create_anonymous_area [1] boot loader reserved area: size 0x2000000
map_backing_store: aspace 0x84a2d000, cache 0x84a41078, offset 0x0, size 2097152, addressSpec 1, wiring 6, protection 48, area 0x80003eb8, areaName 'boot loader reserved area'
vm_create_anonymous_area: done
vm_free_unused_boot_loader_range(): asked to free 0x00000000 - 0xffffefff
free boot range: get rid of 0x00000000 - 0x80000000
free boot range: get rid of 0x80004000 - 0x80014000
free boot range: get rid of 0x80184000 - 0x80185000
free boot range: get rid of 0x80538000 - 0x80880000
free boot range: get rid of 0x80881000 - 0x80882000
free boot range: get rid of 0x80898000 - 0x80a10000
free boot range: get rid of 0x84a10000 - 0x84a1f000
free boot range: get rid of 0x85f1f000 - 0xfe010000
Without keepRanges:
INIT: init VM semaphores
arch_vm_init_end(): 0 virtual ranges to keep:
vm_free_unused_boot_loader_range(): asked to free 0x00000000 - 0xffffefff
free boot range: get rid of 0x00000000 - 0x80000000
free boot range: get rid of 0x80004000 - 0x80014000
free boot range: get rid of 0x80184000 - 0x80185000
free boot range: get rid of 0x80538000 - 0x80880000
free boot range: get rid of 0x80881000 - 0x80882000
free boot range: get rid of 0x80898000 - 0x80a10000
free boot range: get rid of 0x84a10000 - 0x84a1f000
free boot range: get rid of 0x85f1f000 - 0xfe010000
Both hang there.
2010-01-02
Haiku
INIT: init VM semaphores
arch_vm_init_end(): 2 virtual ranges to keep:
start: 0x00132000, size: 0x9000
no kernel address, skipping...
start: 0xff800000, size: 0x200000
create_anonymous_area [1] boot loader reserved area: size 0x200000
map_backing_store: aspace 0x84a2d000, cache 0x84a41078, *vaddr 0xff800000, offset 0x0, size 2097152, addressSpec 1, wiring 6, protection 48, area 0x80003eb8, areaName 'boot loader reserved area'
vm_create_anonymous_area: done
unreserve_boot_loader_ranges()
map_backing_store: aspace 0x84a2d000, cache 0x84a41168, *vaddr 0xccccc000, offset 0x0, size 262144, addressSpec 1, wiring 6, protection 0, area 0x80003f5c, areaName 'uninitialized heap memory'
map_backing_store: aspace 0x84a2d000, cache 0x84a41258, *vaddr 0xdeadb000, offset 0x0, size 262144, addressSpec 1, wiring 6, protection 0, area 0x80003f5c, areaName 'freed heap memory'
INIT: init generic syscall
INIT: init scheduler
scheduler_init: found 1 logical cpu
scheduler_init: using simple scheduler
INIT: init threads
create_anonymous_area [1] undertaker_2_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41348, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003ce8, areaName 'undertaker_2_kstack'
vm_create_anonymous_area: done
INIT: init kernel daemons
create_anonymous_area [1] kernel daemon_3_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41438, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'kernel daemon_3_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] resource resizer_4_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41528, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'resource resizer_4_kstack'
vm_create_anonymous_area: done
INIT: init VM threads
create_anonymous_area [1] page scrubber_5_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41618, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'page scrubber_5_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] page writer_6_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41708, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'page writer_6_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] page daemon_7_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a417f8, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'page daemon_7_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] object cache resizer_8_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a418e8, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d98, areaName 'object cache resizer_8_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] dedicated grow heap: size 0x100000
map_backing_store: aspace 0x84a2d000, cache 0x84a419d8, *vaddr 0x00000000, offset 0x0, size 1048576, addressSpec 5, wiring 2, protection 48, area 0x80003e98, areaName 'dedicated grow heap'
vm_create_anonymous_area: done
heap_add_area: area -1 added to grow heap 0x80900000 - usable range 0x80902000 - 0x80a00000
create_anonymous_area [1] heap grower_9_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41ac8, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003d48, areaName 'heap grower_9_kstack'
vm_create_anonymous_area: done
create_anonymous_area [1] low resource manager_10_kstack: size 0x4000
map_backing_store: aspace 0x84a2d000, cache 0x84a41bb8, *vaddr 0x00000000, offset 0x0, size 16384, addressSpec 4, wiring 2, protection 176, area 0x80003db8, areaName 'low resource manager_10_kstack'
vm_create_anonymous_area: done
2010-01-01
Haiku RemoteDisk
I found out that Haiku has a facility similar to qemu-nbd
that allows to connect to remote disks.
Once I had made the necessary adjustments to make the remote_disk_server run on OpenSolaris,
the client wouldn't discover it. Turns out it didn't set an IP address.
$ jam -q run ':remote_disk_server' haiku.image
Learned some nice things about OpenFirmware, unfortunately it doesn't seem to provide string-to-int conversions -
would've been handy.
2010-01-01
Happy New Year - Life Signs
I managed to get a sign of life from the Haiku kernel - by doing return -42 as first instruction in
_start. Further tries indicate it survives the initial version check and
setting the number of CPUs but gets stuck during the rendezvous.
It's often the small things, in this case an "uninitialized" kernel argument interfering with CPU rendezvous!
kernel entry at 0x8008a24c
kernel stack top: 0x80004000
Welcome to kernel debugger output!
Haiku revision: 34833
INIT: init CPU
INIT: init interrupts
INIT: init VM
head_add_area: area -1 added to small heap 0x8491f000 - usable range 0x8492c000 - 0x8511f000
heap_add_area: area -1 added to medium heap 0x8511f000 - usable range 0x85120000 - 0x855eb000
heap_add_area: area -1 added to large heap 0x855ebccc - usable range 0x8591f000
slab: init base 0x80886000 + 0x2000
reserve_boot_loader_range(): Skipping range: 0x00000000, 12288
reserve_boot_loader_range(): Skipping range: 0x00102000, 180224
reserve_boot_loader_range(): Skipping range: 0x0012f000, 45056
reserve_boot_loader_range(): Skipping range: 0x00400000, 4194304
INIT: init driver_settings
INIT: init notification services
INIT: init teams
INIT: init ELF loader
INIT: init modules
INIT: init semaphores
INIT: init interrupts post vm
exception handlers at 0x80898000
INIT: init system info
INIT: init SMP
smp_init: entry
smp_init: calling arch_smp_init
INIT: init timer
INIT: init real time clock
allocate_commpage_entry(2, 24) -> 0xffff0100
INIT: init condition variables
INIT: init VM semaphores
PANIC: looking up page failed for pa 0x80800000
Welcome to Kernel Debugging Land...
Running on CPU 0
Current thread pointer is 0x801ae3c0, which is an address we can't read from.
kdebug >
2009-12-31
Haiku and OpenFirmware
Testing Haiku on real hardware has been a little painful, requiring to reboot and hold Command+Option+O+F for each try.
While searching for a way to set boot arguments,
by way of the printenv command I've discovered a setting auto-boot?
that saves me the key combination.
setenv auto-boot? false
These options simplify TFTP booting:
setenv default-client-ip 10.0.0.2
setenv default-server-ip 10.0.0.1
Having renamed the bootloader to bl, allowing me to boot by typing:
boot enet:,bl
I can even fully automate this by doing:
setenv auto-boot? true
setenv boot-command boot enet:,bl
2009-12-30
Haiku on ppc
Over Christmas I've been poking some more at Haiku/ppc.
Given an IDE disk to boot from and booting via TFTP, this is where it ends (kernel tracing enabled):
kernel entry at 0x80081de4
kernel stack top: 0x80004000
Here's the output from a modified boot loader
showing the memory mappings right after the output above:
phys memory ranges:
base 0x00000000, length 0x20000000
allocated phys memory ranges:
base 0x00000000, length 0x00101000
base 0x00102000, length 0x0002c000
base 0x0012f000, length 0x00714000
base 0x1fe00000, length 0x00200000
allocated virt memory ranges:
base 0x00000000, length 0x00003000
base 0x00102000, length 0x0002c000
base 0x0012f000, length 0x0000c000
base 0x00400000, length 0x00400000
base 0x80800000, length 0x00081000
base 0x80900000, length 0x00010000
base 0x90000000, length 0x10000000
base 0xfe000000, length 0x00010000
base 0xfec00000, length 0x00001000
base 0xfee00000, length 0x00200000
base 0xff800000, length 0x00200000
base 0x80000000, length 0x00406000
2009-12-20
More ppc fun
haiku-user
I've finally brought
ppc-haiku-user
to a point where it compiles and executes (on Linux and OpenSolaris).
ELF loading isn't finished though.
Haiku/ppc
Following Alex Graf's ppc64/NewWorld patches for QEMU and OpenBIOS (r646), I checked on Haiku/ppc (r34720):
$ qemu-system-ppc -localtime -boot d -cdrom haiku-boot-cd-ppc.iso
checking for memory...
0: base = 0x00000000, size = 134217728
1: empty region
total physical memory = 128 MB
suggested page table size = 1048576
need new page table, size = 1048576!
new table at: 0x07f00000
Booting the CD from the OpenFirmware 3.1.1 prompt on my PowerMac G3 b&w:
0 > boot cd:,\\:tbxi load-size=a5 adler32=96453122
parsing <CHRP-BOOT>
DEFAULT CATCH!, code=6e at %SRR: ff846810 %SRR1: 0000b030
ok
0 >
Better luck using the command line from bootinfo.txt:
0 > boot cd:,\ppc\boot_loader_openfirmware load-size=3ba83 adler32=cac2b42
Loading ELF
checking for memory...
0: base = 0x00000000, size = 134217728
1: base = 0x08000000, size = 134217728
2: empty region
3: base = 0x10000000, size = 134217728
total physical memory = 384 MB
suggested page table size = 4194304
need new page table, size = 4194304!
MSR: 0x00003030
found 12 translations
found exception handlers!
found page table!
virt_allocated: 12
phys_allocated: 5
phys_memory: 1
found CPU: /cpus/PowerPC,750
CPU clock frequency: 300000000
bus clock frequency: 100000000
time base frequency: 25000000
mmu_alloc: va 0x80000000, pa 0x00003000, size 16384
platform_init_heap()
mmu_alloc: va 0x80004000, pa 0x00007000, size 65536
heap base = 0x80004000
heap top = 0x80014000
adding args: ´´
Welcome to the Haiku boot loader!
boot path = "/pci/@d/mac-io/ata-3@20000/disk@0:2,\ppc\boot_loader_openfirmware"
boot type = block
add_partitions_for(0x800004028, mountFS = no)
add_partitions_for(fd = 0, mountFS = no)
0x80004060 Partition::Partition
0x80004060 Partition::Scan()
check for partitioning_system: Amiga Partition Map
check for partitioning_system: Intel Partition Map
check for partitioning_system: Intel Extended Partition
check for partitioning_system: Apple Extended Map
0x80004060 Partition::_Mount check for file_system: BFS Filesystem
0x80004060 Partition::_Mount check for file_system: AmigaFFS Filesystem
0x80004060 Partition::_Mount check for file_system: TAR Filesystem
0x80004060 Partition::~Partition
no boot path found, scan for all partitions...
/pci@80000000/pci-bridge@d/mac-io@5/scsi@10000/disk:0
(failed)
/pci@80000000/pci-bridge@d/mac-io@5/fdc@15000/disk@0:0
NO DRIVE (failed)
/pci@80000000/pci-bridge@d/mac-io@5/ata-3@20000/disk:0
(could open device, handle = 0xff9d0c80, node = 0x80004044)
/pci@80000000/pci-bridge@d/Seri-Tek1S2@3/sd:0
Not surprised that it didn't find a partition, since I didn't provide a haiku.image.
So I waited and waited for OpenSolaris to write a 120 MB image to my USB memory stick...
I found out that Haiku hangs above due to my Sonnet SATA controller. Without it I get to a nice boot menu.
The USB boot volume is not found, but I'd blame the G3 firmware for that.
Consequences of global warming?
Poor Tux!
Something fishy appears to be going on with QEMU's VGABIOS for ppc. Kudos to Andy Warhol.
2009-12-06
Some ppc64 insights from Saint Nick
While last weekend or so Fedora 12 screwed my OpenSolaris installation,
this weekend was more successful.
Inspired by Blue Swirl
hinting me
on QEMU's ppc TCG target as possible cause of my sparc64 troubles,
over at OpenBIOS,
I made an interesting discovery with gdb:
Darwin does not use
function descriptors!
That means ever since
Lupus' mention thereof
I was on the wrong track with my ppc64 port of Mono...
Armed with this insight, I managed to get QEMU working on Mac OS X v10.5 ppc64 yesterday,
and malc applied the patch today. Yay! The road to sending my first proper Git patches was a little stony; I ended up using
nano, not finding switches for supplying the cover letter or the patch version
and not finding a free graphical (copy&paste-capable) text editor that plays nice with git send-email.
Updating Mono to r147748 however, eglib didn't seem to supply G_MAXINT so that the build
was once again broken, and I was too tired to fix it right away.
2009-11-17
DTrace# on i386 Mac
Yesterday I was successful adding support for Mac OS X i386.
Not having access to an x86_64 Mac or sparc hardware, this completes my dtrace-sharp ABI coverage. [1]
Performance was around five seconds on a MacBook Core Duo 2.0 GHz,
so not as bad as ppc but still significantly slower than on Solaris x86.
I figured out that on x86 both Solaris and Mac OS X change a nop instruction
to int13. On the Mac it seemed to be written off-by-one though for the actual probe...
On an unrelated matter,
I finally rebased and uploaded my siginfo patches
for Haiku.
[1] I checked that the latest Git QEMU sparc-softmmu
works again on Mac OS X, unpatched;
however booting a DTrace-capable Solaris still requires sparc64-softmmu and OpenBIOS fixes though.
2009-11-15
More on DTrace performance
Adding support for Solaris amd64 turned out pretty easy. So here are some numbers:
OpenSolaris x86 with probe enabled:
real 0m1,07s
user 0m0,82s
sys 0m0,22s
OpenSolaris x86 with probe disabled:
real 0m1,08s
user 0m0,83s
sys 0m0,22s
OpenSolaris x86 with no probes firing:
real 0m1,05s
user 0m0,81s
sys 0m0,21s
OpenSolaris x86 with no probes firing and without dtrace attached:
real 0m0.103s
user 0m0.085s
sys 0m0.018s
OpenSolaris amd64 with probe enabled:
real 0m1,12s
user 0m0,73s
sys 0m0,20s
OpenSolaris amd64 with probe disabled:
real 0m1,11s
user 0m0,85s
sys 0m0,23s
Since these measurements were not scientifically exact,
I'll regard the absolute difference of 0.01s between disabled and enabled as statistical deviation and thus as "zero impact"
(when the probe was enabled, the D script traced the timestamp).
Leaving out the managed console output, the is-enabled check and the actual probe summed up to around 0.03s.
The test system was an AMD Athlon64 X2 2.2 GHz.
And now for something completely different:
Mac OS X ppc with probe enabled:
real 0m22.077s
user 0m20.188s
sys 0m1.212s
Mac OS X ppc with no probes firing:
real 0m22.114s
user 0m20.137s
sys 0m1.222s
Mac OS X ppc with no probes firing and without dtrace attached:
real 0m0.214s
user 0m0.142s
sys 0m0.046s
If run through dtrace, it spends five times more time in userland and more than 100 times longer in whole!
Without dtrace attached, it comes close to the OpenSolaris numbers.
Test system was a dual-core PowerMac G5 2.0 GHz.
I guess this would be something to investigate with dtrace...
2009-11-15
Breakthrough with DTrace for Mono
Last Tuesday (Nov 12) I finally managed to register a dynamic
DTrace provider
from Mono
on OpenSolaris!
andreas@sunshine:~/Mono/dtrace-sharp$ uname -a
SunOS sunshine 5.11 snv_111b i86pc i386 i86pc Solaris
andreas@sunshine:~/Mono/dtrace-sharp$ dtrace -V
dtrace: Sun D 1.6.2
andreas@sunshine:~/Mono/dtrace-sharp$ /usr/sbin/dtrace -Z -s test.d -c "mono Test.exe"
dtrace: script 'test.d' matched 1 probe
CPU ID FUNCTION:NAME
0 1 :BEGIN 12975497459776
Probe enabled? True
dtrace: pid 1081 has exited
0 72117 :foo 12975697172478
andreas@sunshine:~/Mono/dtrace-sharp$ mono Test.exe
Probe enabled? False
It had been a while since I've actively worked on DTrace support
for the Mono runtime. I integrated
the "mono" provider
in r104964 (June 5, 2008). Since then that allows to trace VES initialization, JIT method compilation and hopefully GCs
via USDT probes - similar to the
Java 1.6 "hotspot" provider.
It was however not yet possible to trace method flow or to define probes at random points in managed code,
because there was simply no documentation.
There was an undocumented libdtrace.so library, but it didn't appear to expose this functionality,
just helpers to run D scripts as used by
the Java DTrace API
(Chime)
or dtrace(1M) itself.
Now a Ruby project gave me a valuable hint: the DTrace Object Format (DOF).
OpenSolaris' sys/dtrace.h header defines a number of structs for encoding a provider definition
in a somewhat platform-neutral format. Then there's a DTRACEHIOC_ADDDOF ioctl
to register such a DOF file with the kernel. The system thereby gets offsets to points in native code,
which it overwrites to make is-enabled probes return 1 if enabled by a client script
and to turn the actual probes from NOPs into a trigger for the client-specified actions.
So what dtrace-sharp does is it lets you specify providers with probes -
with custom provider name, module name, function name and probe name - and registers them with the system.
It emits native stub functions per probe that can be p/invoke'd from managed code when the probe is supposed to fire.
DTrace.Probe myProbe = DTrace.Probe.Create();
myProbe.Name = "foo";
myProbe.Function = "";
DTrace.Provider provider = new DTrace.Provider();
provider.Name = "myprovider";
provider.Probes.Add(myProbe);
provider.Publish();
if (myProbe.IsEnabled)
myProbe.Invoke();
Yesterday I managed the same on Mac OS X ppc.
PMG5:dtrace-sharp andreas$ uname -a
Darwin PMG5.local 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:57:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC Power Macintosh
PMG5:dtrace-sharp andreas$ dtrace -V
dtrace: Sun D 1.2.2
PMG5:dtrace-sharp andreas$ sudo /usr/sbin/dtrace -Z -s test.d -c "mono Test.exe"
dtrace: script 'test.d' matched 1 probe
CPU ID FUNCTION:NAME
1 1 :BEGIN 2086511124199
Probe enabled? True
dtrace: pid 241 has exited
0 16812 somefunc:foo 2101198802905
The dyld source was helpful in figuring out the ioctl,
but who would've guessed that user_addr_t is 8 bytes for a 32-bit userland application?
Also, the sys/dtrace.h header deviates in some places. Apple requires DOF version 3,
which OpenSolaris does not support.
Unfortunately the performance on Mac OS X when running with dtrace was desastrous compared to OpenSolaris...
There are still some limitations of course - the probes can't pass any arguments yet,
and a number of ABIs (Solaris {amd64,sparc[64]}, Mac OS X {i386,x86_64}) are still unsupported.
Back on OpenSolaris, I checked whether the x86 stubs work unmodified on amd64, but no, they crashed.
On to some more disassembling...
So what's ahead? I still need to decide where to push my Git repository once dtrace-sharp becomes usable.
I think it'll be useful even if I continue to add more DTrace support to the Mono runtime,
since being pure managed code it might be usable on other runtimes like GNU Portable.NET as well.
I will be looking into adding method-enter and method-return probes
to the Mono runtime.
To do so I'll try to dump the system-modified is-enabled probe instructions
in order to check whether I can avoid a function call for the is-enabled probe from a hot path like method execution.
