[G Tablet Android Development] rcgabriel:
Okay, minor step - I managed to get into the lag state again and capture the wakeup activity from dmesg. Attached is what happened in dmesg when the system tried to wakeup:

<4>[ 2896.054620] Enabling non-boot CPUs ...
<4>[ 2896.094664] CPU1: Booted secondary processor
<7>[ 2896.094840] CPU0 attaching NULL sched-domain.
<7>[ 2896.257924] CPU0 attaching sched-domain:
<7>[ 2896.257966] domain 0: span 0-1 level CPU
<7>[ 2896.258006] groups: 0 1
<7>[ 2896.258069] CPU1 attaching sched-domain:
<7>[ 2896.258102] domain 0: span 0-1 level CPU
<7>[ 2896.258136] groups: 1 0
<4>[ 2896.258729] CPU1 is up
<4>[ 2896.265833] *** Wakeup from LP1 ***
<6>[ 2896.275836] tegra_uart tegra_uart.1: tegra_uart_resume called
<6>[ 2896.281719] tegra_uart tegra_uart.2: tegra_uart_resume called
<6>[ 2896.287588] tegra_uart tegra_uart.3: tegra_uart_resume called
<6>[ 2896.293450] tegra_uart tegra_uart.4: tegra_uart_resume called
<6>[ 2896.299635] wakeup wake lock: mmc_delayed_work
<4>[ 2896.304965] Resume SDIO2Resume SDIO3switch-dock: dock_switch_work_func
<4>[ 2896.384399] switch-dock: desktop_dock = 0switch-dock: state = 0
<4>[ 2896.390585] h2w: off
<6>[ 2896.536957] usb 1-1: reset high speed USB device using tegra-ehci and address 2
<4>[ 2896.880520] Restarting tasks ...
<6>[ 2896.885505] nvrm_notifier_show: blocking
<4>[ 2896.900670] done.
<6>[ 2896.902777] tegra_pm_notifier: start processing event=4
<6>[ 2896.910332] notify_daemon: wait for nvrm_daemon
<6>[ 2896.933747] nvrm_notifier_show: returning with 'PM_POST_SUSPEND'
<6>[ 2896.947458] nvrm_notifier_show: blocking
<6>[ 2896.960167] tegra_pm_notifier: finished processing event=4
<6>[ 2896.976796] suspend: exit suspend, ret = 0 (2010-12-25 17:58:31.059770041 UTC)
<6>[ 2897.646318] PM: Syncing filesystems ... done.
<6>[ 2897.853777] tegra_pm_notifier: start processing event=3
<6>[ 2897.853814] nvrm_notifier_show: returning with 'PM_SUSPEND_PREPARE'
<6>[ 2897.853997] notify_daemon: wait for nvrm_daemon
<6>[ 2897.857518] nvrm_notifier_show: blocking
<6>[ 2897.857544] tegra_pm_notifier: finished processing event=3
<4>[ 2897.857558] Freezing user space processes ...
<6>[ 2897.862377] nvrm_notifier_show: false wakeup, returning with '\n'
<4>[ 2897.897408] (elapsed 0.03 seconds) done.
<4>[ 2897.901342] Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
<6>[ 2897.946079] tegra_grhost tegra_grhost: suspending
<6>[ 2897.995191] tegra_grhost tegra_grhost: suspended
<4>[ 2898.000022] Suspend SDIO3Suspend SDIO2
<6>[ 2898.003762] tegra_uart tegra_uart.4: tegra_uart_suspend called
<6>[ 2898.009875] tegra_uart tegra_uart.3: tegra_uart_suspend called
<6>[ 2898.015848] tegra_uart tegra_uart.2: tegra_uart_suspend called
<6>[ 2898.021775] tegra_uart tegra_uart.1: tegra_uart_suspend called
<4>[ 2898.029954] DVFS set core at 1200mV
<4>[ 2898.039188] RM power state before suspend: Active (2)
<6>[ 2898.044378] wakeup wake lock: dhd_wake_lock
<4>[ 2898.048660] Active Module: 0x16
<4>[ 2898.060829] Disabling non-boot CPUs ...
<7>[ 2898.064739] CPU0 attaching NULL sched-domain.
<7>[ 2898.064786] CPU1 attaching NULL sched-domain.
<7>[ 2898.166172] CPU0 attaching NULL sched-domain.
<5>[ 2898.166901] CPU1: clean shutdown
<4>[ 2903.940840] Enabling non-boot CPUs ...
<4>[ 2903.985153] CPU1: Booted secondary processor
<7>[ 2903.985351] CPU0 attaching NULL sched-domain.
<7>[ 2904.161206] CPU0 attaching sched-domain:
<7>[ 2904.161247] domain 0: span 0-1 level CPU
<7>[ 2904.161288] groups: 0 1
<7>[ 2904.161350] CPU1 attaching sched-domain:
<7>[ 2904.161383] domain 0: span 0-1 level CPU
<7>[ 2904.161418] groups: 1 0
<4>[ 2904.162007] CPU1 is up
<4>[ 2904.169112] *** Wakeup from LP1 ***
<4>[ 2904.173194] PMU Resume Irq. In 1, 1 --> Send First KEY_F4(LP1) -->Send a simulate KEY_F4 ****
<6>[ 2904.192091] tegra_uart tegra_uart.1: tegra_uart_resume called
<6>[ 2904.205609] tegra_uart tegra_uart.2: tegra_uart_resume called
<6>[ 2904.211490] tegra_uart tegra_uart.3: tegra_uart_resume called
<6>[ 2904.217356] tegra_uart tegra_uart.4: tegra_uart_resume called
<4>[ 2904.389852] LP1(PMU flag set) -->Out.
<4>[ 2904.394220] Resume SDIO2Resume SDIO3h2w: offswitch-dock: dock_switch_work_func
<4>[ 2904.475173] switch-dock: desktop_dock = 0switch-dock: state = 0
<6>[ 2904.909094] usb 1-1: reset high speed USB device using tegra-ehci and address 2
<4>[ 2905.452278] Restarting tasks ...
<6>[ 2905.457281] nvrm_notifier_show: blocking
<4>[ 2905.464230] dhdsdio_htclk: HT Avail request error: -35
<6>[ 2905.470039] request_suspend_state: wakeup (3->0) at 665179051856 (2010-12-25 17:58:39.181279035 UTC)
<4>[ 2905.481490] done.
<6>[ 2905.485183] tegra_pm_notifier: start processing event=4
<6>[ 2905.490567] notify_daemon: wait for nvrm_daemon
<6>[ 2905.506260] nvrm_notifier_show: returning with 'PM_POST_SUSPEND'
<6>[ 2905.515835] request_suspend_state: sleep (0->3) at 665224872861 (2010-12-25 17:58:39.227104040 UTC)
<6>[ 2905.525994] tegra_pm_notifier: finished processing event=4
<6>[ 2905.526044] nvrm_notifier_show: blocking
<6>[ 2905.542803] suspend: exit suspend, ret = 0 (2010-12-25 17:58:39.254074042 UTC)
<6>[ 2909.383679] PM: Syncing filesystems ... done.
<6>[ 2909.570068] tegra_pm_notifier: start processing event=3
<6>[ 2909.570106] nvrm_notifier_show: returning with 'PM_SUSPEND_PREPARE'
<6>[ 2909.570293] notify_daemon: wait for nvrm_daemon
<6>[ 2909.574045] nvrm_notifier_show: blocking
<6>[ 2909.574071] tegra_pm_notifier: finished processing event=3
<4>[ 2909.574085] Freezing user space processes ...
<6>[ 2909.579063] nvrm_notifier_show: false wakeup, returning with '\n'
<4>[ 2909.614570] (elapsed 0.04 seconds) done.
<4>[ 2909.618506] Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
<6>[ 2909.645993] tegra_grhost tegra_grhost: suspending
<6>[ 2909.650737] tegra_grhost tegra_grhost: suspended
<4>[ 2909.655579] Suspend SDIO3Suspend SDIO2
<6>[ 2909.659274] tegra_uart tegra_uart.4: tegra_uart_suspend called
<6>[ 2909.665485] tegra_uart tegra_uart.3: tegra_uart_suspend called
<6>[ 2909.671440] tegra_uart tegra_uart.2: tegra_uart_suspend called
<6>[ 2909.677400] tegra_uart tegra_uart.1: tegra_uart_suspend called
<4>[ 2909.685695] DVFS set core at 1200mV
<4>[ 2909.695809] RM power state before suspend: Active (2)
<4>[ 2909.700946] Active Module: 0x16
<4>[ 2909.713166] Disabling non-boot CPUs ...
<7>[ 2909.717075] CPU0 attaching NULL sched-domain.
<7>[ 2909.717120] CPU1 attaching NULL sched-domain.
<7>[ 2909.823323] CPU0 attaching NULL sched-domain.
<5>[ 2909.824083] CPU1: clean shutdown
<4>[ 2913.962567] Enabling non-boot CPUs ...
<4>[ 2914.005848] CPU1: Booted secondary processor
<7>[ 2914.006025] CPU0 attaching NULL sched-domain.
<7>[ 2914.147290] CPU0 attaching sched-domain:
<7>[ 2914.147332] domain 0: span 0-1 level CPU
<7>[ 2914.147373] groups: 0 1
<7>[ 2914.147437] CPU1 attaching sched-domain:
<7>[ 2914.147469] domain 0: span 0-1 level CPU
<7>[ 2914.147505] groups: 1 0
<4>[ 2914.148092] CPU1 is up
<4>[ 2914.155196] *** Wakeup from LP1 ***
<4>[ 2914.159175] gpio_keys_isr
<4>[ 2914.161937] gpio_keys_isr
<6>[ 2914.169912] tegra_uart tegra_uart.1: tegra_uart_resume called
<6>[ 2914.175793] tegra_uart tegra_uart.2: tegra_uart_resume called
<6>[ 2914.181661] tegra_uart tegra_uart.3: tegra_uart_resume called
<6>[ 2914.187688] tegra_uart tegra_uart.4: tegra_uart_resume called
<4>[ 2914.194418] Resume SDIO2
<6>[ 2914.196870] wakeup wake lock: mmc_delayed_work
<4>[ 2914.202368] Resume SDIO3switch-dock: dock_switch_work_func
<4>[ 2914.298461] switch-dock: desktop_dock = 0h2w: offgpio_keys_isr
<4>[ 2914.327263] switch-dock: state = 0
<4>[ 2914.339183] Restarting tasks ...
<6>[ 2914.355481] nvrm_notifier_show: blocking
<4>[ 2914.379544] done.
<6>[ 2914.381480] tegra_pm_notifier: start processing event=4
<6>[ 2914.397254] nvrm_notifier_show: returning with 'PM_POST_SUSPEND'
<6>[ 2914.403283] notify_daemon: wait for nvrm_daemon
<6>[ 2914.409378] nvrm_notifier_show: blocking
<6>[ 2914.415203] tegra_pm_notifier: finished processing event=4
<6>[ 2914.435267] suspend: exit suspend, ret = 0 (2010-12-25 17:58:48.597119011 UTC)
<6>[ 2914.454982] usb 1-1: reset high speed USB device using tegra-ehci and address 2
<4>[ 2914.481208] gpio_keys_isr
<4>[ 2914.483839] gpio_keys_isr
<4>[ 2914.552830] gpio_keys_isr
<6>[ 2914.695077] PM: Syncing filesystems ... gpio_keys_isr
<4>[ 2914.760382] gpio_keys_isr
<4>[ 2914.876996] gpio_keys_isr
<4>[ 2914.965104] done.
<6>[ 2914.980288] tegra_pm_notifier: start processing event=3
<6>[ 2914.985553] notify_daemon: wait for nvrm_daemon
<6>[ 2914.985567] nvrm_notifier_show: returning with 'PM_SUSPEND_PREPARE'
<6>[ 2914.997591] tegra_pm_notifier: finished processing event=3
<6>[ 2914.997655] nvrm_notifier_show: blocking
<4>[ 2915.008522] Freezing user space processes ...
<6>[ 2915.017796] nvrm_notifier_show: false wakeup, returning with '\n'
<4>[ 2915.024342] (elapsed 0.00 seconds) done.
<4>[ 2915.028285] Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
<4>[ 2915.041537] gpio_keys_isr
<6>[ 2915.074420] tegra_grhost tegra_grhost: suspending
<4>[ 2915.170193] gpio_keys_isr
<4>[ 2915.295102] gpio_keys_isr
<4>[ 2915.337013] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90000
<4>[ 2915.404565] gpio_keys_isr
<6>[ 2915.443428] tegra_grhost tegra_grhost: suspended
<4>[ 2915.448202] Suspend SDIO3Suspend SDIO2
<6>[ 2915.451865] tegra_uart tegra_uart.4: tegra_uart_suspend called
<6>[ 2915.458008] tegra_uart tegra_uart.3: tegra_uart_suspend called
<6>[ 2915.463961] tegra_uart tegra_uart.2: tegra_uart_suspend called
<6>[ 2915.469880] tegra_uart tegra_uart.1: tegra_uart_suspend called
<4>[ 2915.477918] DVFS set core at 1200mV
<3>[ 2915.482118] pm_noirq_op(): platform_pm_suspend_noirq+0x0/0x40 returns -11
<3>[ 2915.488936] PM: Device power.0 failed to suspend late: error -11
<3>[ 2915.495195] PM: Some devices failed to power down
<6>[ 2915.505268] tegra_uart tegra_uart.1: tegra_uart_resume called
<6>[ 2915.511197] tegra_uart tegra_uart.2: tegra_uart_resume called
<6>[ 2915.517116] tegra_uart tegra_uart.3: tegra_uart_resume called
<6>[ 2915.522986] tegra_uart tegra_uart.4: tegra_uart_resume called
<4>[ 2915.529681] Resume SDIO2
<6>[ 2915.532135] wakeup wake lock: mmc_delayed_work
<4>[ 2915.537642] Resume SDIO3gpio_keys_isr
<4>[ 2915.623091] switch-dock: dock_switch_work_func
<4>[ 2915.627735] h2w: offswitch-dock: desktop_dock = 0switch-dock: state = 0
<4>[ 2915.679249] gpio_keys_isr
<4>[ 2915.799761] gpio_keys_isr
<6>[ 2915.826060] usb 1-1: reset high speed USB device using tegra-ehci and address 2
<4>[ 2915.901172] gpio_keys_isr
<4>[ 2916.220150] gpio_keys_isr
<4>[ 2916.315085] gpio_keys_isr
<4>[ 2916.398396] Restarting tasks ...
<6>[ 2916.408347] nvrm_notifier_show: blocking
<4>[ 2916.417953] done.
<6>[ 2916.419922] tegra_pm_notifier: start processing event=4
<6>[ 2916.425395] nvrm_notifier_show: returning with 'PM_POST_SUSPEND'
<6>[ 2916.429205] notify_daemon: wait for nvrm_daemon
<6>[ 2916.437485] tegra_pm_notifier: finished processing event=4
<6>[ 2916.437648] nvrm_notifier_show: blocking
<4>[ 2916.440250] gpio_keys_isr
<6>[ 2916.453371] suspend: exit suspend, ret = 0 (2010-12-25 17:58:50.615197150 UTC)
<6>[ 2916.705173] PM: Syncing filesystems ... done.
<6>[ 2916.902603] tegra_pm_notifier: start processing event=3
<6>[ 2916.902685] nvrm_notifier_show: returning with 'PM_SUSPEND_PREPARE'
<6>[ 2916.903021] notify_daemon: wait for nvrm_daemon
<6>[ 2916.906721] nvrm_notifier_show: blocking
<6>[ 2916.906781] tegra_pm_notifier: finished processing event=3
<4>[ 2916.906813] Freezing user space processes ...
<6>[ 2916.916502] nvrm_notifier_show: false wakeup, returning with '\n'
<4>[ 2916.956909] (elapsed 0.05 seconds) done.
<4>[ 2916.960869] Freezing remaining freezable tasks ... (elapsed 0.00 seconds) done.
<6>[ 2917.006454] tegra_grhost tegra_grhost: suspending
<4>[ 2917.162623] PMU Resume Irq. In 1, 1 --> Send First KEY_F4(LP1) -->Send a simulate KEY_F4 ****
<4>[ 2917.372994] Release quickly(LP1) -->Out.
<6>[ 2917.736102] tegra_grhost tegra_grhost: suspended
<4>[ 2917.741186] Suspend SDIO3Suspend SDIO2
<6>[ 2917.745056] tegra_uart tegra_uart.4: tegra_uart_suspend called
<6>[ 2917.751211] tegra_uart tegra_uart.3: tegra_uart_suspend called
<6>[ 2917.757226] tegra_uart tegra_uart.2: tegra_uart_suspend called
<6>[ 2917.763233] tegra_uart tegra_uart.1: tegra_uart_suspend called
<4>[ 2917.770409] DVFS set core at 1200mV
<3>[ 2917.774674] pm_noirq_op(): platform_pm_suspend_noirq+0x0/0x40 returns -11
<3>[ 2917.781494] PM: Device power.0 failed to suspend late: error -11
<3>[ 2917.787780] PM: Some devices failed to power down
<6>[ 2917.797837] tegra_uart tegra_uart.1: tegra_uart_resume called
<6>[ 2917.803761] tegra_uart tegra_uart.2: tegra_uart_resume called
<6>[ 2917.809633] tegra_uart tegra_uart.3: tegra_uart_resume called
<6>[ 2917.815542] tegra_uart tegra_uart.4: tegra_uart_resume called
<4>[ 2917.822252] Resume SDIO2
<6>[ 2917.824779] wakeup wake lock: mmc_delayed_work
<4>[ 2917.830232] Resume SDIO3switch-dock: dock_switch_work_func
<4>[ 2917.914758] switch-dock: desktop_dock = 0h2w: offswitch-dock: state = 0
<6>[ 2918.072560] usb 1-1: reset high speed USB device using tegra-ehci and address 2
<4>[ 2918.763608] Restarting tasks ...
<6>[ 2918.768570] nvrm_notifier_show: blocking
<4>[ 2918.774787] done.
<6>[ 2918.776305] request_suspend_state: wakeup (3->0) at 674353826619 (2010-12-25 17:58:53.332658039 UTC)
<6>[ 2918.786076] tegra_pm_notifier: start processing event=4
<6>[ 2918.791509] notify_daemon: wait for nvrm_daemon
<6>[ 2918.796158] nvrm_notifier_show: returning with 'PM_POST_SUSPEND'
<6>[ 2918.809484] nvrm_notifier_show: blocking
<6>[ 2918.813546] tegra_pm_notifier: finished processing event=4
<6>[ 2918.819398] suspend: exit suspend, ret = 0 (2010-12-25 17:58:53.375764043 UTC)
<6>[ 2918.829644] nvrm_notifier_show: returning with 'PM_DISPLAY_ON'
<6>[ 2918.836047] notify_daemon: wait for nvrm_daemon
<4>[ 2918.840873] PM_SCREEN_IS_OFF set to 0
<6>[ 2919.188879] nvrm_notifier_show: blocking
<4>[ 2941.617843] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90004
<4>[ 2941.757539] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90000
<4>[ 2957.037550] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90008
<4>[ 2957.167859] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90000
<4>[ 2976.927540] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90004
<4>[ 2977.039658] SO340010: gpio_val=0xc0980000, button_val = 0xdbb90000

I had to press the power button twice to get it to wake up since the first press didn't seem to do anything. You can see it looks like it woke, tried to suspend again, then woke again (if not even a bit more glitchy than that). There's the false wakeup, the error -11 failed to suspend - I don't think those are part of a normal, healthy wakeup procedure.

While not crystal clear to me, I suspect that something visible here, possibly the pm_noirq_op(): platform_pm_suspend_noirq+0x0/0x40 returns -11 is at the heart of the lag issue.

It would be very helpful is somebody else would try taking a dmesg snapshot after a laggy wake-up and see if you see any of the same stuff (from the terminal emulator type dmesg > /sdcard/dmesg.out and then copy the dmesg.out file from your /sdcard area to your desktop).

I'll reboot, re-sleep and see if a second dmesg snapshot looks similar, then compare to a healthy wake-up.