Finished basic implemnation for realtime ingestion