Assume testsample is input() to be df2
can use only requests[security]@2.22.0, beautifulsoup4@4.8.1, pycrypto@2.6.1,
cryptography@2.8, python-dateutil@2.8.1, urllib3@1.25.7,
pyyaml@5.2, numpy@1.16.5, scipy@1.2.2, sympy@1.4,
scikit-learn@0.20.4, nltk@3.4.5, pandas@0.24.2, statsmodels@0.10.2

Input x and y containes NaN values so think abt that also

Description

Close Auction Volume prediction throughout the trading period is essential for various trading stratagies to make critical decisions.

Stock trades on exchange in two phases:

Continuous phase – stock trades in the phase from the start of day till Close Auction takes place. Trades continuously happen in this phase as buyers and sellers agrees on a price. Volume of shares traded in this phase is called Order Book volume.

Close Auction phase – stock trades in this phase at the end of the trading. In this phase large volume is traded at a single price through an auction, we call this volume as close auction volume.

image
Objective

You have to devlop a model which can predict close auction volume based on historic data provided.

Note: You should use training data to come up with a model that predict the Close volume based on historic data.

Use the model that you have developed (based on trainig data), and with the input of test data predict the Close volume for respective symbol_ids.
Training Dataset

You can download the training dataset from here

In the dataset you are given cumulative Order Book volume for various time buckets in a day and close auction volume for that day.

Training dataset is essentially timeseries data for various symbol_ids.
Data Schema

symbol_id: Unique stock identifier.

date_id: Unique date identifier. date_id is sequential.

seconds_bucket: Numbers of seconds elapsed since the start of continuous phase.

cummulative_continous_volume: total volume traded in continuous phase since the start of the continuous phase till the seconds_bucket.

close_volume: volume traded in Close Auction of the stock on that date. This will tick at the end of the day at seconds_bucket = 31200
Evaluation Criteria

Each prediction (xi) will be assessed against the realised Close Auction Volume (yi) for that day using
\text{Score} = 1 - \left( \frac{1}{n} \sum_{i=1}^{n} \frac{|x_i - y_i|}{y_i} \right)

n -> number of symbol_ids

Score for each test case will be added to produce a final score.

Final score on leaderboard and documentation in the code will be used in evaluation.

Note: This problem statement has additional test cases that'll run post your submission. You final score is subject to change, your current score is not the final score
Custom Models

You can upload your models to Github/Gitlab public repository and download the same from your code.

Here is a sample code you can use.

import requests
import os

def download_file(url, filename):
response = requests.get(url)
if response.status_code == 200:
with open(filename, 'wb') as file:
file.write(response.content)
Example usage

url = "https://github.com/sp2hari/hackerrank-network-sample/raw/refs/heads/main/userdata1.parquet"
filename = os.path.basename(url)
download_file(url, filename)

Please make sure you use this to download only model files. Trying to misuse the network access will result in disqualification.

Network access is enabled only for this question.

Input Format

Input will be a filename. This file will be a CSV which contains a timeseries data for various sybmol_ids. Schema is same as training data.

Timeseries starts with date_id = 1.

CSV will contain full data for first 50 days, and for 51st day you will have data till a certain seconds_bucket.

You will need to predict the close volume for 51st day.
How to read input

#!/bin/python3

import pandas as pd

filename = input()
df = pd.read_csv(filename)

Constraints

Predicted Close Volume should be a non-negative number.

Output Format

Output needs to be printed in the format shown in below samples.

Sample Input 0

sample1.csv

Sample Output 0

symbol_id,close_volume
1029677210,1473541.0
1077784782,471017.0
1084320043,3454475.0
1106122882,212836.0

Explanation 0

You can download the Sample1.csv

Sample1.csv has historic data for 4 symbol_ids. For 51st date_id, data is present till seconds_bucket 30600.

Output has predicted Close Volume for those symbol_ids.

Sample Input 1

sample2.csv

Sample Output 1

symbol_id,close_volume
1029677210,3088265.0
1077784782,187838.0
1084320043,2990272.0
1106122882,78281.0

Explanation 1

You can download the Sample2.csv

Sample2.csv has historic data for 4 symbol_ids. For 51st date_id, data is present till seconds_bucket 14400.

Output has predicted Close Volume for those symbol_ids.

Sample Input 2

sample3.csv

Sample Output 2

symbol_id,close_volume
1029677210,2038657.0
1077784782,210855.0
1084320043,4136057.0
1106122882,106879.0

Explanation 2

You can download the Sample3.csv

Sample3.csv has historic data for 4 symbol_ids. For 51st date_id, data is present till seconds_bucket 28500.

Output has predicted Close Volume for those symbol_ids.

fix the below code
right now
Predicted Output (stdout)

symbol_id,close_volume
1029677210,417687.5
1077784782,417687.5
1084320043,417687.5
1106122882,417687.5

Actual Value

symbol_id,close_volume
1029677210,1473541.0
1077784782,471017.0
1084320043,3454475.0
1106122882,212836.0

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error
from io import StringIO
import requests
Assuming you have a download link to the CSV file

url = 'https://github.com/DexterfreaK/GS2/raw/refs/heads/main/training_F.csv'
response = requests.get(url)
csv_data = StringIO(response.text)

import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.impute import SimpleImputer
import sys
----------------------------
Function to handle feature extraction
----------------------------

def extract_features(df, is_train=True):
"""
Extracts the last cumulative_continous_volume for each symbol_id and date_id.

Parameters:
- df (DataFrame): The input dataframe.
- is_train (bool): Flag to indicate if the dataframe is training data.

Returns:
- DataFrame: A dataframe with symbol_id, date_id, and the aggregated feature.
"""
if is_train:
    # For training data, date_id ranges from 1 to 50
    df = df[df['date_id'] <= 50]
else:
    # For test data, date_id is 51
    df = df[df['date_id'] == 51]

# Group by symbol_id and date_id and get the maximum seconds_bucket
idx = df.groupby(['symbol_id', 'date_id'])['seconds_bucket'].idxmax()
last_vol = df.loc[idx].copy()

# Rename the cumulative_continous_volume for clarity
last_vol.rename(columns={'cummulative_continous_volume': 'last_continuous_volume'}, inplace=True)

return last_vol[['symbol_id', 'date_id', 'last_continuous_volume']]

----------------------------
Read Training Data
----------------------------

try:
df1 = pd.read_csv(csv_data)
except FileNotFoundError:
print("Training file 'train.csv' not found.", file=sys.stderr)
sys.exit(1)
Handle NaN values in training data

imputer = SimpleImputer(strategy='median')
df1[['cummulative_continous_volume']] = imputer.fit_transform(df1[['cummulative_continous_volume']])
df1['close_volume'] = df1['close_volume'].fillna(df1['close_volume'].median())
Extract features from training data

train_features = extract_features(df1, is_train=True)
Prepare training inputs and targets

X_train = train_features[['last_continuous_volume']]
y_train = df1[df1['date_id'] <= 50].drop_duplicates(subset=['symbol_id', 'date_id'])['close_volume']
----------------------------
Read Test Data
----------------------------

filename = input().strip()

try:
df2 = pd.read_csv(filename)
except FileNotFoundError:
print(f"Test file '{filename}' not found.", file=sys.stderr)
sys.exit(1)
Handle NaN values in test data using the same imputer fitted on training data

df2[['cummulative_continous_volume']] = imputer.transform(df2[['cummulative_continous_volume']])
Extract features from test data

test_features = extract_features(df2, is_train=False)
Prepare test inputs

X_test = test_features[['last_continuous_volume']]
----------------------------
Model Training
----------------------------

model = LinearRegression()
model.fit(X_train, y_train)
----------------------------
Prediction
----------------------------

predictions = model.predict(X_test)
Ensure non-negative predictions

predictions = np.maximum(predictions, 0)
Round predictions to one decimal place as per the sample output

predictions = predictions.round(1)
----------------------------
Prepare Output
----------------------------

output = pd.DataFrame({
'symbol_id': test_features['symbol_id'],
'close_volume': predictions
})
Sort by symbol_id for consistent output (optional)

output.sort_values('symbol_id', inplace=True)
Print the output in the required format

print("symbol_id,close_volume")
for _, row in output.iterrows():
print(f"{int(row['symbol_id'])},{row['close_volume']}")

create a prediction. model