hstファイルの読み方（C#で

# 目的
C#上でMetaTraderのHSTファイルを扱いたい。
⇒CSVでごにょごにょしたり，C#上で解析できるかも？

# 開発環境
+ Microsoft Visual Studio Community 2019
+ Version 16.11.9
+ Microsoft .NET Framework Version 4.8.09032

# データフォーマットについて

[ここ](http://fatbald.seesaa.net/article/447016624.html)とか[ここ](http://fatbald.seesaa.net/article/438245549.html)が参考になります。

バージョン400とそれ以外で少しフォーマット違うようなので，その辺も考慮します。

最初にはデータのヘッダ部分が含まれていて，この中にバージョン情報も含まれるので，
1. ヘッダを読む
1. ヘッダ情報をもとに400とそれ以外で処理を分ける

ということになると思います。

色々調べた結果，ヘッダー部分は，
``` C#
            int version;
            byte[] copyRight64 = new byte[64];
            string copyRight = "";
            byte[] symbol12 = new byte[12];
            string symbol;
            int period;
            int digits;
            int timeSign, lastSync;


            submitbtn.Text = "実行中";


            byte[] buf;
            using (FileStream fileStream = new FileStream(srcPath.Text, FileMode.Open, FileAccess.Read))
            {
                long size;
                size = fileStream.Length;
                buf = new Byte[size];
                fileStream.Read(buf, 0, (int)size);
            }

            //datasize
            int datasize = (buf.Length - 37 * 4) / 44;

            //バージョンの読込
            version = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //CopyRightの読込
            Buffer.BlockCopy(buf, position, copyRight64, 0, 64);
            copyRight = System.Text.Encoding.UTF8.GetString(copyRight64).TrimEnd('\0');
            position += 64; //64byte

            //Symbolの読込
            Buffer.BlockCopy(buf, position, symbol12, 0, 12);
            symbol = (System.Text.Encoding.UTF8.GetString(symbol12)).TrimEnd('\0');
            position += 12; //12byte

            //Periodの読込
            period = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //Digitsの読込
            digits = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //TimeSignの読込
            timeSign = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //lastSyncの読込
            lastSync = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            position += 4 * 13; //4byte

            string statusStr = "HSTファイルのバージョン:" + version.ToString() + "\r\n" +
                "" + copyRight.Trim() + "\r\n" +
                "通貨ペア:" + symbol.Trim() + "\r\n" +
                "Period:" + period + "\r\n" +
                "Digits:" + digits + "\r\n" +
                "TimeSign:" + (new System.DateTime(1970, 1, 1).AddSeconds(timeSign)).ToString("yyyy/MM/dd-hh:mm") + "\r\n" +
                "LastSync:" + lastSync;
```

で読めるんじゃなかろうかと思います。


# レイアウト

レイアウトはこんな感じにしました。
ボタンを3つ用意して，変換元と変換後，実行ボタンです。

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/531802/47e56735-9f30-53c3-2759-d76ce1e74aad.png)

最終的にはごにょごにょ読み込んで，プログレスバーを更新しながら，CSVに変換したいと思います。


# ソースコード

``` C#

        private string srcPathSelect(int mode)
        {
            if (mode == 0)
            {
                string str;
                srcPath.Text = "";
                OpenFileDialog openFileDialog = new OpenFileDialog();
                openFileDialog.InitialDirectory = "";
                openFileDialog.Filter = "HSTファイル(*.hst)|*.hst|すべてのファイル(*.*)|*.*";
                openFileDialog.FilterIndex = 1;
                openFileDialog.Title = "開くファイルを選択してください";
                openFileDialog.RestoreDirectory = true;

                //ダイアログを表示する
                if (openFileDialog.ShowDialog() == DialogResult.OK)
                    str = openFileDialog.FileName;
                else
                    str = "";
                return str;
            }
            else
            {

                string str;
                srcCSVPath.Text = "";
                OpenFileDialog openFileDialog = new OpenFileDialog();
                openFileDialog.InitialDirectory = "";
                openFileDialog.Filter = "csvファイル(*.csv)|*.csv|すべてのファイル(*.*)|*.*";
                openFileDialog.FilterIndex = 1;
                openFileDialog.Title = "開くファイルを選択してください";
                openFileDialog.RestoreDirectory = true;

                //ダイアログを表示する
                if (openFileDialog.ShowDialog() == DialogResult.OK)
                {
                    str = openFileDialog.FileName;
                    dstHSTpath.Text = Path.GetDirectoryName(openFileDialog.FileName)+"\\"+ Path.GetFileNameWithoutExtension(openFileDialog.FileName) + "_v2.csv";
                }
                else
                    str = "";
                return str;
            }
        }

        private string dstPathSelect(int mode)
        {
            if (mode == 0)
            {
                string str;
                dstPath.Text = "";
                SaveFileDialog saveFileDialog = new SaveFileDialog();
                saveFileDialog.InitialDirectory = "";
                saveFileDialog.Filter = "csvファイル(*.csv)|*.csv";
                saveFileDialog.Title = "保存するファイルを選択してください";
                saveFileDialog.RestoreDirectory = true;
                saveFileDialog.CheckFileExists = false;

                //ダイアログを表示する
                if (saveFileDialog.ShowDialog() == DialogResult.OK)
                    str = saveFileDialog.FileName;

                else
                    str = "";
                return str;
            }
            else
            {
                string str;
                dstPath.Text = "";
                SaveFileDialog saveFileDialog = new SaveFileDialog();
                saveFileDialog.InitialDirectory = "";
                saveFileDialog.Filter = "hstファイル(*.hst)|*.hst";
                saveFileDialog.Title = "保存するファイルを選択してください";
                saveFileDialog.RestoreDirectory = true;
                saveFileDialog.CheckFileExists = false;

                //ダイアログを表示する
                if (saveFileDialog.ShowDialog() == DialogResult.OK)
                    str = saveFileDialog.FileName;
                else
                    str = "";
                return str;
            }
        }

        private void SrcPath_TextChanged(object sender, EventArgs e)
        {

        }

        private void SrcPath_DoubleClick(object sender, EventArgs e)
        {
            srcPath.Text = srcPathSelect(0);
        }

        private void SelectSrcbtn_Click(object sender, EventArgs e)
        {
            srcPath.Text = srcPathSelect(0);
        }

        private void SelectDstbtn_Click(object sender, EventArgs e)
        {
            dstPath.Text = dstPathSelect(0);
        }

        private void DstPath_DoubleClick(object sender, EventArgs e)
        {
            dstPath.Text = dstPathSelect(0);
        }

        private void Submitbtn_Click(object sender, EventArgs e)
        {


            if (srcPath.Text == "") return;
            if (dstPath.Text == "") return;

            int position = 0;
            int version;
            byte[] copyRight64 = new byte[64];
            string copyRight = "";
            byte[] symbol12 = new byte[12];
            string symbol;
            int period;
            int digits;
            int timeSign, lastSync;


            submitbtn.Text = "実行中";


            byte[] buf;
            using (FileStream fileStream = new FileStream(srcPath.Text, FileMode.Open, FileAccess.Read))
            {
                long size;
                size = fileStream.Length;
                buf = new Byte[size];
                fileStream.Read(buf, 0, (int)size);
            }

            //datasize
            int datasize = (buf.Length - 37 * 4) / 44;

            //バージョンの読込
            version = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //CopyRightの読込
            Buffer.BlockCopy(buf, position, copyRight64, 0, 64);
            copyRight = System.Text.Encoding.UTF8.GetString(copyRight64).TrimEnd('\0');
            position += 64; //64byte

            //Symbolの読込
            Buffer.BlockCopy(buf, position, symbol12, 0, 12);
            symbol = (System.Text.Encoding.UTF8.GetString(symbol12)).TrimEnd('\0');
            position += 12; //12byte

            //Periodの読込
            period = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //Digitsの読込
            digits = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //TimeSignの読込
            timeSign = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            //lastSyncの読込
            lastSync = BitConverter.ToInt32(buf, position);
            position += 4; //4byte

            position += 4 * 13; //4byte

            //jikoku 4byte, open 8byte(double) , low 8byte(double) , high 8byte(double) , close 8byte(double) , vol 8byte(double) 全部で44byte
            string statusStr = "HSTファイルのバージョン:" + version.ToString() + "\r\n" +
                "" + copyRight.Trim() + "\r\n" +
                "通貨ペア:" + symbol.Trim() + "\r\n" +
                "Period:" + period + "\r\n" +
                "Digits:" + digits + "\r\n" +
                "TimeSign:" + (new System.DateTime(1970, 1, 1).AddSeconds(timeSign)).ToString("yyyy/MM/dd-hh:mm") + "\r\n" +
                "LastSync:" + lastSync;



            statusBox1.Text = statusStr;


            if (version == 400)
                HST2CSV400(datasize, position, buf);
            else
                HST2CSV401(datasize, position, buf);



            //Console.WriteLine("version:" + version);



        }



        async void HST2CSV400(int datasize, int position, byte[] buf)
        {
            string src = srcPath.Text;
            string dst = dstPath.Text;

            progressBar1.Value = 0;
            progressBar1.Maximum = 100;


            await Task.Run(() =>
            {

                DateTime[] jikoku;
                double[] open;
                double[] low;
                double[] high;
                double[] close;
                double[] vol;

                jikoku = new DateTime[datasize];
                open = new double[datasize];
                low = new double[datasize];
                high = new double[datasize];
                close = new double[datasize];
                vol = new double[datasize];



                using (System.IO.StreamWriter sw = new System.IO.StreamWriter(dst, false, System.Text.Encoding.GetEncoding("shift_jis")))
                {

                    sw.Write("time,open,low,high,close,vol\r\n");
                    for (int i = 0; i < datasize; i++)
                    {
                        jikoku[i] = new System.DateTime(1970, 1, 1).AddSeconds(BitConverter.ToInt32(buf, position));
                        //Buffer.BlockCopy(buf, position, a, 0, 4);
                        //sjikoku[i] = System.Text.Encoding.ASCII.GetString(a);
                        position += 4; //4byte

                      //  sw.Write(jikoku[i].ToString("yyyy/MM/dd-hh:mm") + ",");
                        open[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte

                       // sw.Write(open[i].ToString() + ",");
                        low[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte

                       // sw.Write(low[i].ToString() + ",");
                        high[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte

                        //sw.Write(high[i].ToString() + ",");
                        close[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte

                        //sw.Write(close[i].ToString() + ",");

                        vol[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte

                        //if ((i) % 10 == 0)
                           // progressBarUpdate(100 * i / (datasize));

                       //sw.Write(vol[i].ToString() + "\r\n");
                    }


                }

            });

            progressBarUpdate(100);


            MessageBox.Show("終わりました！");
            submitbtn.Text = "実行";
        }

        async void HST2CSV401(int datasize, int position, byte[] buf)
        {
            string src = srcPath.Text;
            string dst = dstPath.Text;

            progressBar1.Value = 0;
            progressBar1.Maximum = 100;


            await Task.Run(() =>
            {

                DateTime[] jikoku;
                double[] open;
                double[] low;
                double[] high;
                double[] close;
                double[] vol;
                int[] spread;
                double[] rvol;

                jikoku = new DateTime[datasize];
                open = new double[datasize];
                low = new double[datasize];
                high = new double[datasize];
                close = new double[datasize];
                vol = new double[datasize];
                spread = new int[datasize];
                rvol = new double[datasize];



                using (System.IO.StreamWriter sw = new System.IO.StreamWriter(dst, false, System.Text.Encoding.GetEncoding("shift_jis")))
                {

                    sw.Write("time,open,high,low,close,vol,spread,real_volume\r\n");
                    for (int i = 0; i < datasize; i++)
                    {
                        jikoku[i] = new System.DateTime(1970, 1, 1).AddSeconds(BitConverter.ToInt32(buf, position));
                        //Buffer.BlockCopy(buf, position, a, 0, 4);
                        //sjikoku[i] = System.Text.Encoding.ASCII.GetString(a);
                        position += 4; //4byte
                        sw.Write(jikoku[i].ToString("yyyy/MM/dd-hh:mm") + ",");

                        open[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte
                        sw.Write(open[i].ToString() + ",");

                        high[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte
                        sw.Write(high[i].ToString() + ",");

                        low[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte
                        sw.Write(low[i].ToString() + ",");

                        close[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte
                        sw.Write(close[i].ToString() + ",");

                        vol[i] = BitConverter.ToDouble(buf, position);
                        position += 8; //8byte
                        sw.Write(vol[i].ToString() + ",");

                        spread[i] = BitConverter.ToInt32(buf, position);
                        position += 4;
                        sw.Write(spread.ToString() + ",");

                        rvol[i] = BitConverter.ToDouble(buf, position);
                        position += 8;
                        sw.Write(rvol[i].ToString() + "\r\n");

                        if ((i) % 10 == 0)
                            progressBarUpdate(100 * i / (datasize));

                    }


                }

            });

            progressBarUpdate(100);


            MessageBox.Show("終わりました！");
            submitbtn.Text = "実行";
        }

        private void progressBarUpdate(int i)
        {
            if (this.InvokeRequired)
            {
                Invoke(new Action<int>(progressBarUpdate), i);
                return;
            }
            //progressBar1
            progressBar1.Value = i;
            Update();
            //Console.WriteLine(i );

        }

        private void srcCSVbtn_Click(object sender, EventArgs e)
        {
            srcCSVPath.Text = srcPathSelect(1);
        }

        private void dstHSTpath_DoubleClick(object sender, EventArgs e)
        {
            dstHSTpath.Text = dstPathSelect(1);
        }

        private void srcCSVPath_DoubleClick(object sender, EventArgs e)
        {
            srcCSVPath.Text = srcPathSelect(0);
        }

        private void dstHSTBtn_Click(object sender, EventArgs e)
        {
            dstHSTpath.Text = dstPathSelect(0);
        }

        private void exCSV2HSTbtn_Click(object sender, EventArgs e)
        {
            exCSV2HSTbtn.Text = "実行中";
            int counter = 0;
            List<String> lines = new List<string>();
            List<String> hiduke = new List<string>();
            List<String> jikoku = new List<string>();
            List<String> open = new List<string>();
            List<String> close = new List<string>();
            List<String> high = new List<string>();
            List<String> low = new List<string>();
            List<String> vol = new List<string>();
            using (System.IO.StreamReader file = new System.IO.StreamReader(srcCSVPath.Text))
            {
                String line;
                String[] sep = { "," };
                while ((line = file.ReadLine()) != null)
                {
                    String[] val = line.Split(sep, StringSplitOptions.RemoveEmptyEntries);
                    hiduke.Add(val[0]);
                    jikoku.Add(val[1]);
                    open.Add(val[2]);
                    close.Add(val[3]);
                    high.Add(val[4]);
                    low.Add(val[5]);
                    vol.Add(val[6]);
                    counter++;
                }
            }
            StringBuilder sb = new StringBuilder();

            String[] sep2 = { "/" };
            for (int i = 0; i < counter; i++)
            {
                String[] variant = hiduke[i].Split(sep2, StringSplitOptions.RemoveEmptyEntries);
                if (variant.Length < 2)
                    continue;
                sb.Append(variant[2]);
                sb.Append(".");
                sb.Append(variant[0]);
                sb.Append(".");
                sb.Append(variant[1]);
                sb.Append(",");
                sb.Append(jikoku[i].Substring(0, 5));
                sb.Append(",");
                sb.Append(open[i]);
                sb.Append(",");
                sb.Append(close[i]);
                sb.Append(",");
                sb.Append(high[i]);
                sb.Append(",");
                sb.Append(low[i]);
                sb.Append(",");
                sb.Append(vol[i]);
                sb.Append("\r\n");
            }
            // 文字コードを指定
            Encoding enc = System.Text.Encoding.UTF8;

            // ファイルを開く
            using (StreamWriter writer = new StreamWriter(dstHSTpath.Text, false, enc))
            {

                // テキストを書き込む
                writer.WriteLine(sb);

            }


            exCSV2HSTbtn.Text = "実行";

            MessageBox.Show("終わったよん",
    "しゅうりょう～",
    MessageBoxButtons.OK,
    MessageBoxIcon.Asterisk);
        }
```

ソース上で一点注意事項があるとすれば，エクセルとかで読み込む前提だとすると，CSVはShift-Jisにしといた方が良いです。最近のエクセルでは，UTF-8でも読めるようですが・・・。

# REFERENCEs

http://fatbald.seesaa.net/article/447016624.html

https://qiita.com/u1and0/items/6a690f6b0080b8efc2c7

# さいごに

投資は自己責任！

カレンダー参加者募集してます！

https://qiita.com/advent-calendar/2022/mql

Have a good MQL Life!!!