User1: Has anybody already written a script that can look at /etc/init and spit out a graphviz (or similar) dependency DAG ?
User2: User1 like initctl2dot (which we ship by default with Upstart)?
User1: I guess so; I'm still on Ubuntu 10.04 and I hadn't dpkg -L'd anyway
User2: ah, not sure if we already had it back then, but I know it's been around for a while now
User1: I'll just pull it from HEAD and see if it works on its own
User2: if you take the one in trunk, I ported it to python3 which I don't believe we had in 10.04, so I doubt that'll work
User2: taking an older revision should be fine though
User1: OK
User1: http://archive.ubuntu.com/ubuntu/pool/main/u/upstart/upstart_1.8-0ubuntu1.dsc should be old enough to be using python2 ?
User2: there's a pretty good chance 1.8 already had the python3 rewrite as my branch got merged in November 2012
User2: I'm sure that 1.5 is a safer bet
User1: Hm, I can't see an initctl2dot in that tarball
User2: User1 looked under scripts/ ?
User1: I did find -name \*dot
User2: ah, look for dot.py then
User1: Ah found it
User1: I could just run the script on another, newer host
User1: ba-bow: $ ssh spoon initctl show-config -e --err--> initctl: invalid command: show-config
User2: ah, looks like upstart under 10.04 is a bit old for that command...
User2: so probably easier to get a 12.04 machine with the same set of jobs and run initctl2dot on that one
User1: What this is all in aid of, is to work out why squid & an in-house daemon both fail to start sometimes when the server is rebooted
User2: sounds like a race condition between the network interfaces getting configured and the daemon starting
User1: Which I suspect is because they're trying to bind to an interface that isn't up or resolve a hostname before LDAP or DNS is up or something.
User1: Looking at the dependency graph seemed like the best first step :-/
User2: 10.04 wasn't great for that, some stuff were switch to upstart, some were still under sysvinit, so things may be a bit random... I did a fair amount of work in 12.04 to try and get all of that a bit saner (though unfortunately it ended up being way too big to backport to 10.04)
User1: Can you pastebin what initctl show-config -e output looks like?  I can probably mangle that by hand with some sed.  I don't have a newer ubuntu system handy
User2: are squid and your inhouse software using upstart jobs or are they good old sysvinit jobs?
User1: upstart
User2: ok, what are their start conditions?
User1: squid is start on runlevel [2345] or net-device-up IFACE!=lo; stop on runlevel [!2345]
User1: ppcadm is start on runlevel [2345]; stop on runlevel [^2345]
User2: ok, and do those require some special network interfaces to appear? (thinking of bond, bridges, vlan, tunnel devices, ... that kind of stuff)
User1: That first one looks buggy to me, looks to me like it assumes a single-NIC system
User1: Lemme pastebin the interfaces(5)
User2: output of initctl showconfig -e on 13.10: http://paste.ubuntu.com/6152669/
User1: http://paste.debian.net/45811/
User1: AFAIK we don't have bonding on anything anymore, at least one with no bonding is having startup issues
User2: ok and upstream, staff and prisoner are all aliases for real network interfaces setup through udev?
User1: Yeah they're renamed from eth0 style names
User2: my current guess is that those appear as standard ethX devices which then get renamed by udev, so you get a first net-device-up event with the wrong name before it vanishes to get renamed...
User1: Ah.
User2: User1 you could try to get both jobs to do "start on net-device-up IFACE=prisoner" or whatever interface they actually need
User1: So if I just slap in an "AND net-device-up IFACE=staff AND net-device-up IFACE=prisoner" ?
User2: yep, that'd work, upstart will just wait until it got all 3 events and then will start the service
User1: I don't really care if they don't start for two minutes or whatever after boot -- plan Z is to just add a great big "pre-start exec sleep 300"
User2: sounds like having upstart wait for the interfaces would be considerably cleaner than a huge sleep in pre-start ;)
User1: No argument there
User1: The problem is that atm after a power outage, upstart tries a few times then gives up, end result is they never actually start and then the inmates riot because they can't get their porn.
User1: I'll do some restarts with extra net-device-up
User1: Hm, syslog says squid is getting an ABRT.
User1: (Before I change the start on)
User1: Ah, the reason squid is crashing is because some fool thought it would be a good idea to make it use a whitelist helper that talks to postgres.  And postgres gjoseph@example.org is in sysvinit still.
User1: Fun :-/
User1: And I'm guessing "start on runlevel [2345]" is when that runlevel *begins*, not when it ends.
User1: And the ppcadm one is not starting because it needs slapd to be accepting connections before it starts.
User1: Which is also in sysvinit gjoseph@example.org
User3: Anyone written a rule for when `sudo service <service> restart` is invoked? http://upstart.ubuntu.com/cookbook/#restart
User3: I'm writing an upstart job to daemonize a script, and am struggling to capture when `restart` is called -- what state is the process in?
User3: is there an upstart condition? It's not pre-start or pre-stop
User4: User3 is the problem that your job is not able to restart the daemon script? Have you read http://upstart.ubuntu.com/cookbook/#expect ?
User3: I can restart it just fine
User3: that works
User3: but I want to capture in the daemons .conf file WHEN that happens, do FOo
User3: Foo*
User4: User3 there is no "restart" state - using the restart(8) command, the job is restarted whilst retaining its original config. Is this job a system-level one? If so, if you really want the job itself to know it has restarted, your best bet it probably to log something via a pre-start then check that value in the pre-start so you know how many times you've been restarted.
User3: User4 hmm. OK.
User4: User3 generally jobs shouldn't care if they have been restarted - they just need to handle that scenario
User3: yeah I'll have to leave some breadcrumbs and look for them
User3: I need to fire off an email in the event that a daemon was restarted, started, or stopped
User3: as I care about the state of a daemon on a headless server
User4: User3 it would be better then to create a new "monitor" job that specifies "start on (started or stopped)" and does whatever you need.
User5: Hi. How do I analyze the startup process of my Ubuntu 12.04 server?
User5: Is there an equivalent to `systemd-analyze`?
User4: User5 not atm. I suggest you apt-get install bootchart: http://www.bootchart.org/
User5: User4 Thanks. I already tried it, but it is not conclusive yet as a mdraid resync runs in the background.
User5: A little off topic, is there a guide what I can disable on a server (where no graphics is needed for example).
User5: I am on Ubuntu 12.04. How do I easily find out if there are Upstart scripts in later releases for packages not shipping them?
User5: Collectd is installed and does not seem to have one.
User5: Mark Brooks published one here http://blog.loadbalancer.org/upstart-script-for-collectd/ and I wonder if Ubuntu already ships one so I can compare them.
User4: User5 Some of this is indeed rather off-topic (you are probably better off asking some of this on #ubuntu-server in future). However, you can use http://packages.ubuntu.com to query package contents for any ubuntu release. Or use 'apt-file list <pkg>' on a newer version of Ubuntu (you'll need to 'apt-get install apt-file') to see what files a package ships.
User5: User4 Thanks. (That method is not very user-friendly though. ;-) )
User4: User5 ?
User5: User4 Looking at http://packages.ubuntu.com/saucy/amd64/collectd-core/filelist and finding out that there is an upstart file, I would not be able to easily check the content of the file or copy it.
User5: In the end I would need to download the deb file and extract it, if I am not mistaken.
User5: Or are all packageâs source files tracked in Launchpad in Bazaar?
User4: User5 yes, you can access the files on bzr: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/collectd/saucy/files
User5: User4 Awesome! Thanks. That method is easy!
User5: Though in this case I could not find the upstart job.
User5: http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/collectd/saucy/files/head:/debian/
User6: Hello, upstart newbie here. I have a server in single user mode and want to disable the starting of an upstart service for a few restarts, then enable it again.
User6: *upstart 0.6.5
User6: :(
upstart
Ubuntu IRC