Repository: getsentry/sentry
Pull request: (onpremise) Fix `TypeError: a float is required` in 0024_auto_20191230_2052.py
Fixes: https://github.com/getsentry/onpremise/issues/366

**Description of bug**
If an on-premise user has migrated rapidly (within the past 90 days) from 9.0.0 to 9.1.2 to 10.x they may encounter a `TypeError: a float is required` error because the event.timestamp does not exist and None is not a valid posix timestamp. 


The fix I've implemented wraps event.datetime in a try except block, and if it gets a type error uses the recieved_timestamp instead

**Alternative approaches considered:**
Something I considered, which may be cleaner from a maintainability perspective is overriding the insert logic in the migration, that way you don't have this try except block in a widely used code path



Problem statement:
Upgrade 9.1.2 to 10. Migration error
Hello!
I've started to use sentry from version 9.0.0. On prod running sentry 9.1.2 now.
I decided install stage with onpremise. Also migrated database from prod. Version 9.1.2 works fine.
When upgrading to 10 I've got an error during migration 0024_auto_20191230_2052:
```
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, jira_ac, nodestore, sentry, sessions, sites, social_auth
Running migrations:
  Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-10-29 11:37:43.582226) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-01-27 11:37:43.582226) while time zone support is active.
  RuntimeWarning)
Events to process: 131482

Traceback (most recent call last):
  File "/usr/local/bin/sentry", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/__init__.py", line 164, in main
    cli(prog_name=get_prog(), obj={}, max_content_width=100)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py", line 30, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 168, in upgrade
    _upgrade(not noinput, traceback, verbosity, not no_repair)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 130, in _upgrade
    ignore_ghost_migrations=True,
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/sentry/new_migrations/monkey/executor.py", line 26, in apply_migration
    state, migration, fake=fake, fake_initial=fake_initial
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 70, in backfill_eventstream
    primary_hash = event.get_primary_hash()
AttributeError: 'Event' object has no attribute 'get_primary_hash'
```

How can I fix it?

Hints from issue discussion:
Heya, thanks for reporting - we are looking into this.
I've got one more question.
I used Ansible for deploying all of the sentry's services.
What is more correct: to build an image from this Docker file https://github.com/getsentry/sentry/blob/10.0.0/docker/Dockerfile or use a prepared image from dockerhub getsentry/sentry:10.0.0 ?
@artful88533 [forums](https://forum.sentry.io) are a better place to discuss these kind of questions ;)
I'm having this issue too,
@BYK thank you for the fix, can you please tell when this fix will be available in DockerHub?

```shell
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, jira_ac, nodestore, sentry, sessions, sites, social_auth
Running migrations:
  Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-10-31 14:34:41.644628) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-01-29 14:34:41.644628) while time zone support is active.
  RuntimeWarning)
Events to process: 6459

Traceback (most recent call last):
  File "/usr/local/bin/sentry", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/__init__.py", line 164, in main
    cli(prog_name=get_prog(), obj={}, max_content_width=100)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py", line 30, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 168, in upgrade
    _upgrade(not noinput, traceback, verbosity, not no_repair)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 130, in _upgrade
    ignore_ghost_migrations=True,
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/sentry/new_migrations/monkey/executor.py", line 26, in apply_migration
    state, migration, fake=fake, fake_initial=fake_initial
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 109, in backfill_eventstream
    skip_consume=True,
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/services.py", line 104, in <lambda>
    context[key] = (lambda f: lambda *a, **k: getattr(self, f)(*a, **k))(key)
  File "/usr/local/lib/python2.7/site-packages/sentry/eventstream/snuba.py", line 114, in insert
    "message": event.search_message,
AttributeError: 'Event' object has no attribute 'search_message'
```
@bushari - the fix for the original issue is already on Docker Hub. Yours seem to be a different error. I'll look into it. 
@BYK Ok, thank you...
Not sure if it's the same bug, but I've got yet another exception message from the same migration. When I've tested the upgrade procedure a few days ago it worked.

```py
[...]
  Applying sentry.0023_hide_environment_none_20191126... OK
  Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-10-31 21:05:15.196611) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-01-29 21:05:15.196611) while time zone support is active.
  RuntimeWarning)
Events to process: 104594

Traceback (most recent call last):
  File "/usr/local/bin/sentry", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/__init__.py", line 164, in main
    cli(prog_name=get_prog(), obj={}, max_content_width=100)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py", line 30, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 168, in upgrade
    _upgrade(not noinput, traceback, verbosity, not no_repair)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 130, in _upgrade
    ignore_ghost_migrations=True,
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/sentry/new_migrations/monkey/executor.py", line 26, in apply_migration
    state, migration, fake=fake, fake_initial=fake_initial
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 96, in backfill_eventstream
    primary_hash = _get_primary_hash(event)
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 69, in _get_primary_hash
    for hash in (x.get_hash() for x in _get_grouping_variants(event).values())
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 62, in _get_grouping_variants
    return get_grouping_variants_for_event(event, config)
  File "/usr/local/lib/python2.7/site-packages/sentry/grouping/api.py", line 196, in get_grouping_variants_for_event
    components = _get_calculated_grouping_variants_for_event(event, config)
  File "/usr/local/lib/python2.7/site-packages/sentry/grouping/api.py", line 143, in _get_calculated_grouping_variants_for_event
    rv = strategy.get_grouping_component_variants(event, config=config)
  File "/usr/local/lib/python2.7/site-packages/sentry/grouping/strategies/base.py", line 122, in get_grouping_component_variants
    component = self.get_grouping_component(event, variant, config)
  File "/usr/local/lib/python2.7/site-packages/sentry/grouping/strategies/base.py", line 98, in get_grouping_component
    iface = event.interfaces.get(iface_path)
AttributeError: 'Event' object has no attribute 'interfaces'
Cleaning up...
```
It seems that there are some problems with that migration file, i tried to do a clean install on a different machine and i got a small error but most of the migration prcesses been done

Here is the error of sentry.0024_auto_20191230_2052
```shell
Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-10-31 15:57:56.376485) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-01-29 15:57:56.376485) while time zone support is active.
  RuntimeWarning)
Nothing to do, skipping migration.
```

Here is the full migration process
```shell
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, jira_ac, nodestore, sentry, sessions, sites, social_auth
Running migrations:
  Applying sentry.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying jira_ac.0001_initial... OK
  Applying nodestore.0001_initial... OK
  Applying sentry.0002_912_to_recent... OK
  Applying sentry.0003_auto_20191022_0122... OK
  Applying sentry.0004_bitfieldtestmodel_blankjsonfieldtestmodel_callabledefaultmodel_jsonfieldtestmodel_jsonfieldwithdefau... OK
  Applying sentry.0005_fix_content_types... OK
  Applying sentry.0006_sentryapp_date_published... OK
  Applying sentry.0007_auto_20191029_0131... OK
  Applying sentry.0008_auto_20191030_0016... OK
  Applying sentry.0009_auto_20191101_1608... OK
  Applying sentry.0010_auto_20191104_1641... OK
  Applying sentry.0011_remove_pagerdutyservice_service_id_from_state... OK
  Applying sentry.0012_remove_pagerdutyservice_service_id... OK
  Applying sentry.0013_auto_20191111_1829... OK
  Applying sentry.0014_delete_sentryappwebhookerror... OK
  Applying sentry.0015_delete_sentryappwebhookerror_db... OK
  Applying sentry.0016_delete_alert_rule_deprecated_fields... OK
  Applying sentry.0017_incident_aggregation... OK
  Applying sentry.0018_discoversavedquery_version... OK
  Applying sentry.0019_auto_20191114_2040... OK
  Applying sentry.0020_auto_20191125_1420... OK
  Applying sentry.0021_auto_20191203_1803... OK
  Applying sentry.0021_auto_20191202_1716... OK
  Applying sentry.0022_merge... OK
  Applying sentry.0023_hide_environment_none_20191126... OK
  Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-10-31 15:57:56.376485) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-01-29 15:57:56.376485) while time zone support is active.
  RuntimeWarning)
Nothing to do, skipping migration.

 OK
  Applying sentry.0025_organizationaccessrequest_requester... OK
  Applying sentry.0026_delete_event... OK
  Applying sentry.0027_exporteddata... OK
  Applying sessions.0001_initial... OK
  Applying sites.0001_initial... OK
  Applying sites.0002_alter_domain_unique... OK
  Applying social_auth.0001_initial... OK
```
Hey all, @lynnagara and I are looking into this. The issue only affects people migrating from 9.1.2, fresh installs are unaffected. Although we hope to resolve the issue by EOD today, you can use `SENTRY_IMAGE=getsentry/sentry:b0ea6ea032d886edf24a70c1d1ac0d79e1462b39 ./install.sh` (this is using a slightly older version of Sentry) to upgrade and then switch to latest afterwards in the meantime.
Hi,

I was able to successfully upgrade our Sentry instance using the older version (https://github.com/getsentry/sentry/commit/b0ea6ea032d886edf24a70c1d1ac0d79e1462b39). Before doing that I've tried the upgrade Sentry using the latest available version (https://github.com/getsentry/sentry/commit/1e17bb461d26999c5cd89f6f2e274923ce0010d9) but ran into another bug:

```py
  Applying sentry.0022_merge... OK
  Applying sentry.0023_hide_environment_none_20191126... OK
  Applying sentry.0024_auto_20191230_2052.../usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2019-11-03 19:06:53.816964) while time zone support is active.
  RuntimeWarning)
/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Event.datetime received a naive datetime (2020-02-01 19:06:53.816964) while time zone support is active.
  RuntimeWarning)
Events to process: 106367

Traceback (most recent call last):
  File "/usr/local/bin/sentry", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/__init__.py", line 164, in main
    cli(prog_name=get_prog(), obj={}, max_content_width=100)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py", line 30, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 168, in upgrade
    _upgrade(not noinput, traceback, verbosity, not no_repair)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/upgrade.py", line 130, in _upgrade
    ignore_ghost_migrations=True,
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/sentry/new_migrations/monkey/executor.py", line 26, in apply_migration
    state, migration, fake=fake, fake_initial=fake_initial
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/sentry/migrations/0024_auto_20191230_2052.py", line 85, in backfill_eventstream
    skip_consume=True,
  File "/usr/local/lib/python2.7/site-packages/sentry/utils/services.py", line 104, in <lambda>
    context[key] = (lambda f: lambda *a, **k: getattr(self, f)(*a, **k))(key)
  File "/usr/local/lib/python2.7/site-packages/sentry/eventstream/snuba.py", line 116, in insert
    "datetime": event.datetime,
  File "/usr/local/lib/python2.7/site-packages/sentry/eventstore/models.py", line 101, in datetime
    date = datetime.fromtimestamp(timestamp)
TypeError: a float is required
Cleaning up...
```

Commit message: Move the timestamp call to a callback

Patch to review:
diff --git a/src/sentry/migrations/0024_auto_20191230_2052.py b/src/sentry/migrations/0024_auto_20191230_2052.py
index 8653ff6faa8de3..8ddfe3bc730bc2 100644
--- a/src/sentry/migrations/0024_auto_20191230_2052.py
+++ b/src/sentry/migrations/0024_auto_20191230_2052.py
@@ -55,6 +55,18 @@ def _attach_related(_events):
             event.group = groups.get(event.group_id)
         eventstore.bind_nodes(_events, "data")
 
+    def _attach_timestamp(_events):
+        """
+        When migrating old data from Sentry 9.0.0 to 9.1.2 to 10 in rapid succession, the event timestamp may be missing.
+        This function exists to attach the timestamp if its missing
+        """
+        for event in _events:
+            if not event.data.data.get("timestamp"):
+                # Python 2.7 way to convert datetime to timestamp.
+                timestamp = ((e.datetime.replace(tzinfo=None) - e.datetime.utcoffset()) - datetime(1970, 1, 1)).total_seconds()
+                e.data.data['timestamp'] = timestamp
+        eventstore.bind_nodes(_events, "data")
+
     if skip_backfill:
         print("Skipping backfill.\n")
         return
@@ -69,7 +81,7 @@ def _attach_related(_events):
     print("Events to process: {}\n".format(count))
 
     processed = 0
-    for e in RangeQuerySetWrapper(events, step=100, callbacks=(_attach_related,)):
+    for e in RangeQuerySetWrapper(events, step=100, callbacks=(_attach_related, _attach_timestamp,)):
         event = NewEvent(
             project_id=e.project_id, event_id=e.event_id, group_id=e.group_id, data=e.data.data
         )


Code review comments:
diff_hunk: @@ -55,6 +55,18 @@ def _attach_related(_events):
             event.group = groups.get(event.group_id)
         eventstore.bind_nodes(_events, "data")
 
+    def _attach_timestamp(_events):
+        """
+        When migrating old data from Sentry 9.0.0 to 9.1.2 to 10 in rapid succession, the event timestamp may be missing.
+        This function exists to attach the timestamp if its missing
+        """
+        for event in _events:
+            if not event.data.data.get("timestamp"):
+                # Python 2.7 way to convert datetime to timestamp.
+                timestamp = ((e.datetime.replace(tzinfo=None) - e.datetime.utcoffset()) - datetime(1970, 1, 1)).total_seconds()
+                e.data.data['timestamp'] = timestamp
path: src/sentry/migrations/0024_auto_20191230_2052.py
text: @user1:
Don't think `e` is defined here?

@author:
Ah you're right. I had forgotten to update this after I moved it into the callback

diff_hunk: @@ -55,6 +55,18 @@ def _attach_related(_events):
             event.group = groups.get(event.group_id)
         eventstore.bind_nodes(_events, "data")
 
+    def _attach_timestamp(_events):
+        """
+        When migrating old data from Sentry 9.0.0 to 9.1.2 to 10 in rapid succession, the event timestamp may be missing.
+        This function exists to attach the timestamp if its missing
+        """
+        for event in _events:
+            if not event.data.data.get("timestamp"):
+                # Python 2.7 way to convert datetime to timestamp.
path: src/sentry/migrations/0024_auto_20191230_2052.py
text: @user1:
FYI we are working towards a migration to Python 3.6 and beyond so I hope this works in them too?

@author:
Yes, this works going forward as well. The main consideration is that python 3.3+ has a function that converts a datetime to a timestamp without doing the ugliness below. 

@user2:
I think you can leverage the `to_timestamp` helper here (and remove the python 2.7 comment) https://github.com/getsentry/sentry/blob/master/src/sentry/utils/dates.py#L15-L23

@author:
Ah perfect! I'll update now and then run through a test cycle

diff_hunk: @@ -69,7 +81,7 @@ def _attach_related(_events):
     print("Events to process: {}\n".format(count))
 
     processed = 0
-    for e in RangeQuerySetWrapper(events, step=100, callbacks=(_attach_related,)):
+    for e in RangeQuerySetWrapper(events, step=100, callbacks=(_attach_related, _attach_timestamp,)):
path: src/sentry/migrations/0024_auto_20191230_2052.py
text: @user1:
Any reason for not bundling this into `_attach_related`?

@author:
I've updated this per your suggestion so we avoid yet another loop through the events.