Your call on whether you like this more -- please excuse any broken SQL oddities:
```sql
SELECT MAX(datefield) FROM (
    (SELECT COALESCE(idea_idea.time, date('2001-01-01')) AS datefield)
    UNION (SELECT submit_date FROM django_comments WHERE content_type_id = %s and object_pk = idea_idea.id)
    UNION (SELECT time FROM idea_vote WHERE  idea_id = idea_idea.id)
) AS recent_activity
```