Repository: simonmichael/hledger
Pull request #2328: run: cache all journal files, allow commands to use -f, and other improvements
Author: login: adept
Labels: name: A-WISH
description: Some kind of improvement request or proposal.
name: run
Created: 2025-02-22T19:05:54Z | Merged: 2025-03-06T21:54:00Z

Further improvement of 'run'.

- Now commands given to `run` could have `-f` argument and all those journals would be cached and read only once.

- `run` supports numeric arguments (`-3`, this was actually already true, just needed a doc fix) and @ARGFILEs

- `run` can call `run` 


Diff:
diff --git a/doc/FILES.md b/doc/FILES.md
index 7879c00c490..039d611a017 100644
--- a/doc/FILES.md
+++ b/doc/FILES.md
@@ -449,6 +449,7 @@ src/hledger/
           README.md
           Register.md
           Rewrite.md
+          Repl.md
           Roi.md
           Run.md
           Stats.md
diff --git a/hledger-lib/Hledger/Data/Balancing.hs b/hledger-lib/Hledger/Data/Balancing.hs
index 97eeaf89089..49c1183de19 100644
--- a/hledger-lib/Hledger/Data/Balancing.hs
+++ b/hledger-lib/Hledger/Data/Balancing.hs
@@ -64,7 +64,7 @@ data BalancingOpts = BalancingOpts
   , infer_balancing_costs_ :: Bool  -- ^ Are we permitted to infer missing costs to balance transactions ?
                                     --   Distinct from InputOpts{infer_costs_}.
   , commodity_styles_      :: Maybe (M.Map CommoditySymbol AmountStyle)  -- ^ commodity display styles
-  } deriving (Show)
+  } deriving (Eq, Ord, Show)
 
 defbalancingopts :: BalancingOpts
 defbalancingopts = BalancingOpts
diff --git a/hledger-lib/Hledger/Data/Types.hs b/hledger-lib/Hledger/Data/Types.hs
index 22a794e39b6..afd666558ba 100644
--- a/hledger-lib/Hledger/Data/Types.hs
+++ b/hledger-lib/Hledger/Data/Types.hs
@@ -666,7 +666,7 @@ data SepFormat
   = Csv  -- comma-separated
   | Tsv  -- tab-separated
   | Ssv  -- semicolon-separated
-  deriving Eq
+  deriving (Eq, Ord)
 
 -- | The id of a data format understood by hledger, eg @journal@ or @csv@.
 -- The --output-format option selects one of these for output.
@@ -677,7 +677,7 @@ data StorageFormat
   | Timeclock 
   | Timedot 
   | Sep SepFormat 
-  deriving Eq
+  deriving (Eq, Ord)
 
 instance Show SepFormat where
   show Csv = "csv"
diff --git a/hledger-lib/Hledger/Read/InputOptions.hs b/hledger-lib/Hledger/Read/InputOptions.hs
index 1657a9595df..1c57e653d8c 100644
--- a/hledger-lib/Hledger/Read/InputOptions.hs
+++ b/hledger-lib/Hledger/Read/InputOptions.hs
@@ -44,7 +44,7 @@ data InputOpts = InputOpts {
     ,strict_            :: Bool                 -- ^ do extra correctness checks ?
     ,_defer             :: Bool                 -- ^ internal flag: postpone checks, because we are processing multiple files ?
     ,_ioDay             :: Day                  -- ^ today's date, for use with forecast transactions  XXX this duplicates _rsDay, and should eventually be removed when it's not needed anymore.
- } deriving (Show)
+ } deriving (Eq, Ord, Show)
 
 definputopts :: InputOpts
 definputopts = InputOpts
diff --git a/hledger/Hledger/Cli.hs b/hledger/Hledger/Cli.hs
index 314558c0a5b..ed25d3525ad 100644
--- a/hledger/Hledger/Cli.hs
+++ b/hledger/Hledger/Cli.hs
@@ -429,8 +429,9 @@ main = withGhcDebug' $ do
           ensureJournalFileExists . NE.head =<< journalFilePathFromOpts opts
           withJournalDo opts (cmdaction opts)
 
-        -- 6.5.4. run needs findBuiltinCommands passed to it to avoid circular dependency in the code
-        | cmdname == "run" -> do withJournalDo opts $ Hledger.Cli.Commands.Run.run findBuiltinCommand opts
+        -- 6.5.4. "run" and "repl" need findBuiltinCommands passed to it to avoid circular dependency in the code
+        | cmdname == "run"  -> Hledger.Cli.Commands.Run.run Nothing findBuiltinCommand opts
+        | cmdname == "repl" -> Hledger.Cli.Commands.Run.repl findBuiltinCommand opts
 
         -- 6.5.5. all other builtin commands - read the journal and if successful run the command with it
         | otherwise -> withJournalDo opts $ cmdaction opts
diff --git a/hledger/Hledger/Cli/CliOptions.hs b/hledger/Hledger/Cli/CliOptions.hs
index 1d0a677ea6a..64af3be4af1 100644
--- a/hledger/Hledger/Cli/CliOptions.hs
+++ b/hledger/Hledger/Cli/CliOptions.hs
@@ -67,6 +67,7 @@ module Hledger.Cli.CliOptions (
   -- * CLI option accessors
   -- | These do the extra processing required for some options.
   journalFilePathFromOpts,
+  journalFilePathFromOptsNoDefault,
   rulesFilePathFromOpts,
   outputFileFromOpts,
   outputFormatFromOpts,
@@ -696,12 +697,20 @@ getHledgerCliOpts mode' = do
 -- File paths can have a READER: prefix naming a reader/data format.
 journalFilePathFromOpts :: CliOpts -> IO (NE.NonEmpty String)
 journalFilePathFromOpts opts = do
-  f <- defaultJournalPath
+  mbpaths <- journalFilePathFromOptsNoDefault opts
+  case mbpaths of
+    Just paths -> return paths
+    Nothing -> do
+      f <- defaultJournalPath
+      return $ NE.fromList [f]
+
+-- | Like journalFilePathFromOpts, but does not use defaultJournalPath
+journalFilePathFromOptsNoDefault :: CliOpts -> IO (Maybe (NE.NonEmpty String))
+journalFilePathFromOptsNoDefault opts = do
   d <- getCurrentDirectory
-  maybe
-    (return $ NE.fromList [f])
-    (mapM (expandPathPreservingPrefix d))
-    $ NE.nonEmpty $ file_ opts
+  case NE.nonEmpty $ file_ opts of
+    Nothing -> return Nothing
+    Just paths -> Just <$> mapM (expandPathPreservingPrefix d) paths
 
 expandPathPreservingPrefix :: FilePath -> PrefixedFilePath -> IO PrefixedFilePath
 expandPathPreservingPrefix d prefixedf = do
diff --git a/hledger/Hledger/Cli/Commands.hs b/hledger/Hledger/Cli/Commands.hs
index 485f24b2d88..0167e042abd 100644
--- a/hledger/Hledger/Cli/Commands.hs
+++ b/hledger/Hledger/Cli/Commands.hs
@@ -136,7 +136,8 @@ builtinCommands = [
   ,(registermode           , register)
   ,(rewritemode            , rewrite)
   ,(roimode                , roi)
-  ,(runmode                , run')
+  ,(runmode                , runOrReplStub)
+  ,(replmode               , runOrReplStub)
   ,(statsmode              , stats)
   ,(tagsmode               , tags)
   ,(testmode               , testcmd)
@@ -226,6 +227,7 @@ commandsList progversion othercmds =
   ,""
     -----------------------------------------80-------------------------------------
   ,bold' "USER INTERFACES (alternate UIs)"
+  ," repl                     run interactive REPL"
   ,"+ui                       run terminal UI"                                       -- hledger-ui
   ,"+web                      run web UI"                                            -- hledger-web
                                                                                      -- see also: MoLe, https://hledger.org/mobile.html
diff --git a/hledger/Hledger/Cli/Commands/Repl.md b/hledger/Hledger/Cli/Commands/Repl.md
new file mode 100644
index 00000000000..e088f8a0bc3
--- /dev/null
+++ b/hledger/Hledger/Cli/Commands/Repl.md
@@ -0,0 +1,44 @@
+## repl
+
+Runs hledger commands interactively.
+
+This command is EXPERIMENTAL and could change in the future.
+
+```flags
+Flags:
+no command-specific flags
+```
+
+This command starts a read-eval-print loop (REPL) where you can enter commands interactively. To exit REPL, use "exit" or "quit", or send EOF.
+
+It could also accept commands from standard input, if you pipe commands into it.
+
+The commands will run more quickly than if run individually, because the input files would be parsed only once.
+
+Syntax of the commands is intentionally simple:
+- each line is a single hledger command
+- empty lines are skipped
+- everything after `#` is considered to be a comment and will be ignored, and will not be printed out
+- `echo <text>` will print out `<text>`
+
+You can use single or double quotes to quote arguments that need it ('like this' or "like this").
+
+### Caveats:
+
+- `Repl`, like any other command, will load the input file(s) (specified by `LEDGER_FILE` or by `-f` arguments). The contents of those files would be used by all the commands that `repl` runs. If you want a particular command to use a different input file, you can use `-f` flag for that particular command. This will override (not add) the input for that particular command. All the input files would be cached, and would be read only once.
+
+### Examples:
+
+To start the REPL:
+```cli
+hledger repl
+```
+or
+```cli
+hledger repl -f some.journal
+```
+
+To pipe commands into REPL:
+```cli
+(echo "files"; echo "stats") | hledger repl -f some.journal
+```
diff --git a/hledger/Hledger/Cli/Commands/Repl.txt b/hledger/Hledger/Cli/Commands/Repl.txt
new file mode 100644
index 00000000000..6a524e332af
--- /dev/null
+++ b/hledger/Hledger/Cli/Commands/Repl.txt
@@ -0,0 +1,49 @@
+repl
+
+Runs hledger commands interactively.
+
+This command is EXPERIMENTAL and could change in the future.
+
+Flags:
+no command-specific flags
+
+This command starts a read-eval-print loop (REPL) where you can enter
+commands interactively. To exit REPL, use "exit" or "quit", or send EOF.
+
+It could also accept commands from standard input, if you pipe commands
+into it.
+
+The commands will run more quickly than if run individually, because the
+input files would be parsed only once.
+
+Syntax of the commands is intentionally simple: - each line is a single
+hledger command - empty lines are skipped - everything after # is
+considered to be a comment and will be ignored, and will not be printed
+out - echo <text> will print out <text>
+
+You can use single or double quotes to quote arguments that need it
+('like this' or "like this").
+
+Caveats:
+
+-   Repl, like any other command, will load the input file(s) (specified
+    by LEDGER_FILE or by -f arguments). The contents of those files
+    would be used by all the commands that repl runs. If you want a
+    particular command to use a different input file, you can use -f
+    flag for that particular command. This will override (not add) the
+    input for that particular command. All the input files would be
+    cached, and would be read only once.
+
+Examples:
+
+To start the REPL:
+
+hledger repl
+
+or
+
+hledger repl -f some.journal
+
+To pipe commands into REPL:
+
+(echo "files"; echo "stats") | hledger repl -f some.journal
diff --git a/hledger/Hledger/Cli/Commands/Run.hs b/hledger/Hledger/Cli/Commands/Run.hs
index f34cf1b65d2..bb8276bef95 100644
--- a/hledger/Hledger/Cli/Commands/Run.hs
+++ b/hledger/Hledger/Cli/Commands/Run.hs
@@ -12,23 +12,35 @@ The @run@ command allows you to run multiple commands via REPL or from the suppl
 module Hledger.Cli.Commands.Run (
   runmode
  ,run
- ,run'
+ ,replmode
+ ,repl
+ ,runOrReplStub
 ) where
 
+import qualified Data.List.NonEmpty as NE
+import qualified Data.Map.Strict as Map
+import Data.Semigroup (sconcat)
 import qualified Data.Text as T
 import qualified Data.Text.IO as T
 import System.Console.CmdArgs.Explicit as C ( Mode )
 import Hledger
 import Hledger.Cli.CliOptions
 
+import Control.Exception
+import Control.Concurrent.MVar
 import Control.Monad (forM_)
 import Control.Monad.IO.Class (liftIO)
 import Control.Monad.Extra (concatMapM)
 
-import System.Directory (doesFileExist)
+import System.Exit (ExitCode)
+import System.Console.CmdArgs.Explicit (expandArgsAt, modeNames)
+import System.IO (stdin, hIsTerminalDevice, hIsOpen)
+import System.IO.Unsafe (unsafePerformIO)
 import System.Console.Haskeline
 
 import Safe (headMay)
+import Hledger.Cli.DocFiles (runTldrForPage, runInfoForTopic, runManForTopic)
+import Hledger.Cli.Utils (journalTransform)
 
 -- | Command line options for this command.
 runmode = hledgerCommandMode
@@ -38,41 +50,74 @@ runmode = hledgerCommandMode
   )
   cligeneralflagsgroups1
   hiddenflags
-  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...]")
+  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...] OR [-- command1 args... -- command2 args... -- command3 args...]")
 
--- | The fake run command introduced to break circular dependency
-run' :: CliOpts -> Journal -> IO ()
-run' _opts _j = return ()
+replmode = hledgerCommandMode
+  $(embedFileRelative "Hledger/Cli/Commands/Repl.txt")
+  (
+  []
+  )
+  cligeneralflagsgroups1
+  hiddenflags
+  ([], Nothing)
+
+-- | The fake run/repl command introduced to break circular dependency.
+-- This module needs access to `findBuiltinCommand`, which is defined in Hledger.Cli.Commands
+-- However, Hledger.Cli.Commands imports this module, which creates circular dependency.
+-- We expose this do-nothing function so that it could be included in the list of all commands inside
+-- Hledger.Cli.Commands and ensure that "run" is recognized as a valid command by the Hledger.Cli top-level
+-- command line parser. That parser, however, would not call run'. It has a special case for "run", and
+-- will call "run" (see below), passing it `findBuiltinCommand`, thus breaking circular dependency.
+runOrReplStub :: CliOpts -> Journal -> IO ()
+runOrReplStub _opts _j = return ()
+
+-- | Default input files that would be used by commands if
+--   there is no explicit alternative given
+newtype DefaultRunJournal = DefaultRunJournal (NE.NonEmpty String) deriving (Show)
 
 -- | The actual run command.
-run :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> CliOpts -> Journal -> IO ()
-run findBuiltinCommand CliOpts{rawopts_=rawopts} j = do
-  let args = dbg1 "args" $ listofstringopt "args" rawopts
-  case args of
-    [] -> runREPL findBuiltinCommand j
-    maybeFile:_ -> do
-      -- Check if arguments could be interpreted as files.
-      -- If not, assume that they are files
-      isFile <- doesFileExist maybeFile
-      case isFile of
-        True  -> runFromFiles findBuiltinCommand args j
-        False -> runFromArgs  findBuiltinCommand args j
-
-runFromFiles :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> Journal -> IO ()
-runFromFiles findBuiltinCommand inputfiles j = do
+run :: Maybe DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> CliOpts -> IO ()
+run defaultJournalOverride findBuiltinCommand cliopts@CliOpts{rawopts_=rawopts} = do
+  withJournalCached defaultJournalOverride cliopts $ \(_,key) -> do
+    let args = dbg1 "args" $ listofstringopt "args" rawopts
+    isTerminal <- isStdinTerminal
+    if args == [] && not isTerminal
+      then do
+        inputFiles <- journalFilePathFromOpts cliopts
+        let journalFromStdin = any (== "-") $ map (snd . splitReaderPrefix) $ NE.toList inputFiles
+        if journalFromStdin
+        then error' "'run' can't read commands from stdin, as one of the input files was stdin as well"
+        else runREPL key findBuiltinCommand
+      else do
+        -- Check if arguments start with "--".
+        -- If not, assume that they are files with commands
+          case args of
+            "--":_ -> runFromArgs  key findBuiltinCommand args
+            _ -> runFromFiles key findBuiltinCommand args
+
+-- | The actual repl command.
+repl :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> CliOpts -> IO ()
+repl findBuiltinCommand cliopts = do
+  withJournalCached Nothing cliopts $ \(_,key) -> do
+    runREPL key findBuiltinCommand
+
+-- | Run commands from files given to "run".
+runFromFiles :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runFromFiles defaultJournalOverride findBuiltinCommand inputfiles = do
   dbg1IO "inputfiles" inputfiles
   -- read commands from all the inputfiles
   commands <- (flip concatMapM) inputfiles $ \f -> do
     dbg1IO "reading commands" f
     lines . T.unpack <$> T.readFile f
 
-  forM_ commands (runCommand findBuiltinCommand j . parseCommand)
+  forM_ commands (runCommand defaultJournalOverride findBuiltinCommand . parseCommand)
 
-runFromArgs :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> Journal -> IO ()
-runFromArgs findBuiltinCommand args j = do
+-- | Run commands from command line arguments given to "run".
+runFromArgs :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runFromArgs defaultJournalOverride findBuiltinCommand args = do
   -- read commands from all the inputfiles
   let commands = dbg1 "commands from args" $ splitAtElement "--" args
-  forM_ commands (runCommand findBuiltinCommand j)
+  forM_ commands (runCommand defaultJournalOverride findBuiltinCommand)
 
 -- When commands are passed on the command line, shell will parse them for us
 -- When commands are read from file, we need to split the line into command and arguments
@@ -81,32 +126,130 @@ parseCommand line =
   -- # begins a comment, ignore everything after #
   takeWhile (not. ((Just '#')==) . headMay) $  words' (strip line)
 
-runCommand :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> [String] -> IO ()
-runCommand findBuiltinCommand j cmdline = do
-  dbg1IO "running command" cmdline
-  -- # begins a comment, ignore everything after #
+-- | Take a single command line (from file, or REPL, or "--"-surrounded block of the args), and run it.
+runCommand :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runCommand defaultJournalOverride findBuiltinCommand cmdline = do
+  dbg1IO "runCommand for" cmdline
   case cmdline of
     "echo":args -> putStrLn $ unwords $ args
     cmdname:args ->
       case findBuiltinCommand cmdname of
-      Nothing -> putStrLn $ unwords (cmdname:args)
-      Just (cmdmode,cmdaction) -> do
-        opts <- getHledgerCliOpts' cmdmode args
-        cmdaction opts j
+        Nothing -> error' $ "Unrecognized command: " ++ unwords (cmdname:args)
+        Just (cmdmode,cmdaction) -> do
+              -- Even though expandArgsAt is done by the Cli.hs, it stops at the first '--', so we need
+              -- to do it here as well to make sure that each command can use @ARGFILEs
+              args' <- replaceNumericFlags <$> expandArgsAt args
+              dbg1IO "runCommand final args" (cmdname,args')
+              opts <- getHledgerCliOpts' cmdmode args'
+              let
+                rawopts      = rawopts_ opts
+                mmodecmdname = headMay $ modeNames cmdmode
+                helpFlag     = boolopt "help"    rawopts
+                tldrFlag     = boolopt "tldr"    rawopts
+                infoFlag     = boolopt "info"    rawopts
+                manFlag      = boolopt "man"     rawopts
+              if
+                | helpFlag  -> runPager $ showModeUsage cmdmode ++ "\n"
+                | tldrFlag  -> runTldrForPage $ maybe "hledger" (("hledger-"<>)) mmodecmdname
+                | infoFlag  -> runInfoForTopic "hledger" mmodecmdname
+                | manFlag   -> runManForTopic "hledger"  mmodecmdname
+                | otherwise -> do
+                  withJournalCached (Just defaultJournalOverride) opts $ \(j,key) -> do
+                    if cmdname == "run" -- allow "run" to call "run"
+                      then run (Just key) findBuiltinCommand opts
+                      else cmdaction opts j
     [] -> return ()
 
-runREPL :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> IO ()
-runREPL findBuiltinCommand j = do
-  putStrLn "Enter hledger commands, or 'help' for help."
-  runInputT defaultSettings loop
+-- | Run an interactive REPL.
+runREPL :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> IO ()
+runREPL defaultJournalOverride findBuiltinCommand = do
+  isTerminal <- isStdinTerminal
+  if not isTerminal
+    then runInputT defaultSettings (loop "")
+    else do
+      putStrLn "Enter hledger commands. To exit, enter 'quit' or 'exit', or send EOF."
+      runInputT defaultSettings (loop "% ")
   where
-  loop :: InputT IO ()
-  loop = do
-    minput <- getInputLine "% "
+  loop :: String -> InputT IO ()
+  loop prompt = do
+    minput <- getInputLine prompt
     case minput of
       Nothing -> return ()
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
-        loop
+        liftIO $ (runCommand defaultJournalOverride findBuiltinCommand $ parseCommand input)
+                  `catches`
+                  [Handler (\(e::ErrorCall) -> putStr $ show e)
+                  ,Handler (\(_::ExitCode) -> return ())
+                  ,Handler (\UserInterrupt -> return ())
+                  ]
+        loop prompt
+
+isStdinTerminal = do
+  op <- hIsOpen stdin
+  if op then hIsTerminalDevice stdin else return False
+
+-- | Cache of all journals that have been read by commands given to "run",
+-- keyed by the fully-expanded filename.
+journalCache :: MVar (Map.Map (InputOpts,PrefixedFilePath) Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+{-# NOINLINE journalCache #-}
+
+-- | Cache of stdin contents, so that we can re-read it if InputOptions change
+stdinCache :: MVar (Maybe T.Text)
+stdinCache = unsafePerformIO $ newMVar Nothing
+{-# NOINLINE stdinCache #-}
+
+-- | Similar to `withJournal`, but uses caches all the journals it reads.
+-- When reading from stdin, caches the stdin contents so that we could reprocess
+-- it if a read with different InputOptions is requested.
+withJournalCached :: Maybe DefaultRunJournal -> CliOpts -> ((Journal, DefaultRunJournal) -> IO ()) -> IO ()
+withJournalCached defaultJournalOverride cliopts cmd = do
+  (j,key) <- case defaultJournalOverride of
+    Nothing -> journalFilePathFromOpts cliopts >>= readFiles
+    Just (DefaultRunJournal defaultFiles) -> do
+      mbjournalpaths <- journalFilePathFromOptsNoDefault cliopts
+      case mbjournalpaths of
+        Nothing -> readFiles defaultFiles -- use the journal(s) given to the "run" itself
+        Just journalpaths -> readFiles journalpaths
+  cmd (j,key)
+  where
+    readFiles journalpaths = do
+      j <- journalTransform cliopts . sconcat <$> mapM (readAndCacheJournalFile (inputopts_ cliopts)) journalpaths
+      return (j, DefaultRunJournal journalpaths)
+    -- | Read a journal file, caching it (and InputOptions used to read it) if it has not been seen before.
+    -- If the same file is requested with different InputOptions, we read it anew and cache
+    -- it separately.
+    readAndCacheJournalFile :: InputOpts -> PrefixedFilePath -> IO Journal
+    readAndCacheJournalFile iopts fp = do
+      dbg1IO "readAndCacheJournalFile" fp
+      modifyMVar journalCache $ \cache ->
+        case Map.lookup (ioptsWithoutReportSpan,fp) cache of
+          Just journal -> do
+            dbg1IO "readAndCacheJournalFile using cache" (fp, iopts)
+            return (cache, journal)
+          Nothing -> do
+            dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
+            either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
+      where
+        -- InputOptions contain reportspan_ that is used to calculare forecast period,
+        -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
+        -- For the purposes of caching, we want to ignore this, as it is only used for forecast
+        -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
diff --git a/hledger/Hledger/Cli/Commands/Run.md b/hledger/Hledger/Cli/Commands/Run.md
index 3bce630ee51..8af0f66f786 100644
--- a/hledger/Hledger/Cli/Commands/Run.md
+++ b/hledger/Hledger/Cli/Commands/Run.md
@@ -1,6 +1,6 @@
 ## run 
 
-Runs a sequence of hledger commands on the same input file(s), either interactively or as a script.
+Runs a sequence of hledger commands on the same input file(s), taking them from the command line or from file(s).
 
 This command is EXPERIMENTAL and syntax could change in the future.
 
@@ -12,20 +12,21 @@ no command-specific flags
 The commands will run more quickly than if run individually, because the input files would be parsed only once.
 
 "run" has three ways of invocation:
-- when invoked without arguments, it start a read-eval-print loop (REPL) where you can enter commands interactively. To exit REPL, use "exit" or "quit", or send EOF.
 
-- when file names are given to "run", it will read commands from these files, in order.
+- when all positional arguments of "run" are valid file names, "run" will read commands from these files, in order: `run -f some.journal file1.txt file2.txt file3.txt`.
 
-- lastly, commands could be specified directly on the command line. All commands (including the very first one) should be preceded by argument "--"
+- commands could be specified directly on the command line. All commands (including the very first one) should be preceded by argument "--": `run -f some.journal -- cmd1 -- cmd2 -- cmd3`.
 
-Syntax of the commands (either in the file, or in REPL) is intentionally simple:
-- each line is a single hledger command
-- lines that can't be interpreted as hledger commands are printed out as-is
+- it could also accept commands from standard input, if you don't provide an positional arguments and pipe commands into it.
+
+Syntax of the command is intentionally simple:
+- each line read from a file is a single hledger command
 - empty lines are skipped
 - everything after `#` is considered to be a comment and will be ignored, and will not be printed out
-- `echo <text>` will print out text, even if it could be recognized as a hledger command 
+- `echo <text>` will print out `<text>`
+- `run` is a valid command to use as well, so you can have `run` call `run` if you want to.
 
-You can use single quotes or double quotes to quote aguments that need quoting.
+You can use single or double quotes to quote arguments that need it ('like this' or "like this").
 
 You can use `#!/usr/bin/env hledger run` in the first line of the file to make it a runnable script. If this complains about "binary `hledger run` not found", use `/usr/bin/env -S hledger run`.
 
@@ -33,30 +34,34 @@ You can use `#!/usr/bin/env hledger run` in the first line of the file to make i
 
 - If you meant to provide file name as an argument, but made a mistake and a gave file name that does not exist, "run" will attempt to interpret it as a command.
 
-- Numeric flags like `-3` do not work, use long form `--depth 3`
+- `Run`, like any other command, will load the input file(s) (specified by `LEDGER_FILE` or by `-f` arguments). The contents of those files would be used by all the commands that `run` runs. If you want a particular command to use a different input file, you can use `-f` flag for that particular command. This will override (not add) the input for that particular command. All the input files would be cached, and would be read only once.
 
 ### Examples:
 
-To start the REPL:
+Pipe commands into `run`, one per line:
 ```cli
-hledger run
+(echo "files"; echo "stats") | hledger repl -f some.journal
 ```
 
-To provide commands on the command line, separate them with `--`:
+To provide commands on the command line, separate them with `--` (including the first one):
 ```cli
-hledger run -f some.journal -- balance assets --depth 2 -- balance liabilities --depth 3 --transpose
+hledger run -f some.journal -- balance assets --depth 2 -- balance liabilities -f /some/other.journal --depth 3 --transpose -- stats
 ```
+This would load `some.journal`, run `balance assets --depth 2` on it, then run `balance liabilities --depth 3 --transpose` on `/some/other.journal`, and finally will run `stats` on `some.journal`
 
 To provide commands in the file, as a runnable scripts:
 ```cli
-#!/usr/bin/env -S hledger run
-echo "List of accounts"
+#!/usr/bin/env -S hledger run -f some.journal
+echo "List of accounts in some.journal"
 accounts
 
-echo "Assets"
+echo "Assets of some.journal"
 balance assets --depth 2
 
-echo "Liabilities"
-balance liabilities --depth 3 --transpose
+echo "Liabilities from /some/other.journal"
+balance liabilities -f /some/other.journal --depth 3 --transpose
+
+echo "Commands from another.script, applied to another.journal"
+run -f another.journal another.script
 ```
 
diff --git a/hledger/Hledger/Cli/Commands/Run.txt b/hledger/Hledger/Cli/Commands/Run.txt
index 23d9c088fe2..537fc85ad12 100644
--- a/hledger/Hledger/Cli/Commands/Run.txt
+++ b/hledger/Hledger/Cli/Commands/Run.txt
@@ -1,7 +1,7 @@
 run
 
-Runs a sequence of hledger commands on the same input file(s), either
-interactively or as a script.
+Runs a sequence of hledger commands on the same input file(s), taking
+them from the command line or from file(s).
 
 This command is EXPERIMENTAL and syntax could change in the future.
 
@@ -11,26 +11,27 @@ no command-specific flags
 The commands will run more quickly than if run individually, because the
 input files would be parsed only once.
 
-"run" has three ways of invocation: - when invoked without arguments, it
-start a read-eval-print loop (REPL) where you can enter commands
-interactively. To exit REPL, use "exit" or "quit", or send EOF.
+"run" has three ways of invocation:
 
--   when file names are given to "run", it will read commands from these
-    files, in order.
+-   when all positional arguments of "run" are valid file names, "run"
+    will read commands from these files, in order:
+    run -f some.journal file1.txt file2.txt file3.txt.
 
--   lastly, commands could be specified directly on the command line.
-    All commands (including the very first one) should be preceded by
-    argument "--"
+-   commands could be specified directly on the command line. All
+    commands (including the very first one) should be preceded by
+    argument "--": run -f some.journal -- cmd1 -- cmd2 -- cmd3.
 
-Syntax of the commands (either in the file, or in REPL) is intentionally
-simple: - each line is a single hledger command - lines that can't be
-interpreted as hledger commands are printed out as-is - empty lines are
-skipped - everything after # is considered to be a comment and will be
-ignored, and will not be printed out - echo <text> will print out text,
-even if it could be recognized as a hledger command
+-   it could also accept commands from standard input, if you don't
+    provide an positional arguments and pipe commands into it.
 
-You can use single quotes or double quotes to quote aguments that need
-quoting.
+Syntax of the command is intentionally simple: - each line read from a
+file is a single hledger command - empty lines are skipped - everything
+after # is considered to be a comment and will be ignored, and will not
+be printed out - echo <text> will print out <text> - run is a valid
+command to use as well, so you can have run call run if you want to.
+
+You can use single or double quotes to quote arguments that need it
+('like this' or "like this").
 
 You can use #!/usr/bin/env hledger run in the first line of the file to
 make it a runnable script. If this complains about "binary hledger run
@@ -42,26 +43,40 @@ Caveats:
     and a gave file name that does not exist, "run" will attempt to
     interpret it as a command.
 
--   Numeric flags like -3 do not work, use long form --depth 3
+-   Run, like any other command, will load the input file(s) (specified
+    by LEDGER_FILE or by -f arguments). The contents of those files
+    would be used by all the commands that run runs. If you want a
+    particular command to use a different input file, you can use -f
+    flag for that particular command. This will override (not add) the
+    input for that particular command. All the input files would be
+    cached, and would be read only once.
 
 Examples:
 
-To start the REPL:
+Pipe commands into run, one per line:
+
+(echo "files"; echo "stats") | hledger repl -f some.journal
 
-hledger run
+To provide commands on the command line, separate them with --
+(including the first one):
 
-To provide commands on the command line, separate them with --:
+hledger run -f some.journal -- balance assets --depth 2 -- balance liabilities -f /some/other.journal --depth 3 --transpose -- stats
 
-hledger run -f some.journal -- balance assets --depth 2 -- balance liabilities --depth 3 --transpose
+This would load some.journal, run balance assets --depth 2 on it, then
+run balance liabilities --depth 3 --transpose on /some/other.journal,
+and finally will run stats on some.journal
 
 To provide commands in the file, as a runnable scripts:
 
-#!/usr/bin/env -S hledger run
-echo "List of accounts"
+#!/usr/bin/env -S hledger run -f some.journal
+echo "List of accounts in some.journal"
 accounts
 
-echo "Assets"
+echo "Assets of some.journal"
 balance assets --depth 2
 
-echo "Liabilities"
-balance liabilities --depth 3 --transpose
+echo "Liabilities from /some/other.journal"
+balance liabilities -f /some/other.journal --depth 3 --transpose
+
+echo "Commands from another.script, applied to another.journal"
+run -f another.journal another.script
diff --git a/hledger/hledger.cabal b/hledger/hledger.cabal
index 1fdf2e871a3..7ae6ea53c9c 100644
--- a/hledger/hledger.cabal
+++ b/hledger/hledger.cabal
@@ -84,6 +84,7 @@ extra-source-files:
     Hledger/Cli/Commands/Print.txt
     Hledger/Cli/Commands/Register.txt
     Hledger/Cli/Commands/Rewrite.txt
+    Hledger/Cli/Commands/Repl.txt
     Hledger/Cli/Commands/Roi.txt
     Hledger/Cli/Commands/Run.txt
     Hledger/Cli/Commands/Stats.txt
diff --git a/hledger/package.yaml b/hledger/package.yaml
index 0f48c9ddff6..3c395528b51 100644
--- a/hledger/package.yaml
+++ b/hledger/package.yaml
@@ -83,6 +83,7 @@ extra-source-files:
 - Hledger/Cli/Commands/Print.txt
 - Hledger/Cli/Commands/Register.txt
 - Hledger/Cli/Commands/Rewrite.txt
+- Hledger/Cli/Commands/Repl.txt
 - Hledger/Cli/Commands/Roi.txt
 - Hledger/Cli/Commands/Run.txt
 - Hledger/Cli/Commands/Stats.txt
diff --git a/hledger/test/run.args b/hledger/test/run.args
new file mode 100644
index 00000000000..d638d87c494
--- /dev/null
+++ b/hledger/test/run.args
@@ -0,0 +1,2 @@
+--depth=1
+assets
diff --git a/hledger/test/run.test b/hledger/test/run.test
new file mode 100644
index 00000000000..e45ad5f08af
--- /dev/null
+++ b/hledger/test/run.test
@@ -0,0 +1,151 @@
+# * run command
+
+# ** 0. Run requires -- before first command
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- aregister cash
+>2 /hledger: aregister: openFile: does not exist \(No such file or directory\)/
+>=1
+
+# ** 0. Run refuses to read input file and commands from stdin
+<
+$ hledger run -f-
+>2 /hledger: Error: 'run' can't read commands from stdin, as one of the input files was stdin as well/
+>=1
+
+# ** 0. Run one command from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- register cash
+>
+2017-01-01 groceries            assets:cash                  $-100         $-100
+
+
+# ** 0. Run two commands from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- echo "Cash:" -- register cash -- echo 'Food:' -- register food
+>
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+
+# ** 0. Run two commands from command line with different input options
+<
+2017-01-01 investment
+   assets:cash  -$100
+   expenses:food
+
+~ 2017-02-01  next month groceries
+   assets:cash  -$120
+   expenses:food
+$ hledger run -f- -- register cash -- echo '=====' -- register --forecast food
+>
+2017-01-01 investment           assets:cash                  $-100         $-100
+=====
+2017-01-01 investment           expenses:food                 $100          $100
+2017-02-01 next month grocer..  expenses:food                 $120          $220
+
+# ** 0. Run commands from file
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- run_cmds.txt
+>
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+
+# ** 0. Run commands from command line, and use nested "run" to run commands from file
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- echo "Accounts:" -- accounts -- run run_cmds.txt -- echo "Balance:" -- balance cash
+>
+Accounts:
+assets:cash
+expenses:food
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+Balance:
+               $-100  assets:cash
+--------------------
+               $-100  
+
+# ** 0. Run commands from stdin
+<
+echo 'Accounts:'
+accounts
+echo
+echo 'Assets:'
+balance assets
+$ hledger run -f sample.journal
+>
+Accounts:
+assets:bank:checking
+assets:bank:saving
+assets:cash
+expenses:food
+expenses:supplies
+income:gifts
+income:salary
+liabilities:debts
+
+Assets:
+                  $1  assets:bank:saving
+                 $-2  assets:cash
+--------------------
+                 $-1  
+
+
+# ** 0. Run commands accept numeric options
+<
+echo 'Assets, depth 1:'
+balance assets -1
+echo
+echo 'Assets, depth 2:'
+balance assets -2
+$ hledger run -f sample.journal
+>
+Assets, depth 1:
+                 $-1  assets
+--------------------
+                 $-1  
+
+Assets, depth 2:
+                  $1  assets:bank
+                 $-2  assets:cash
+--------------------
+                 $-1  
+
+# ** 0. Run commands accept @ARGFILEs
+<
+echo 'Assets, depth 1:'
+balance @run.args
+echo
+echo 'Assets, depth 2:'
+balance assets -2
+$ hledger run -f sample.journal
+>
+Assets, depth 1:
+                 $-1  assets
+--------------------
+                 $-1  
+
+Assets, depth 2:
+                  $1  assets:bank
+                 $-2  assets:cash
+--------------------
+                 $-1  
diff --git a/hledger/test/run_cmds.txt b/hledger/test/run_cmds.txt
new file mode 100644
index 00000000000..d977b927ee2
--- /dev/null
+++ b/hledger/test/run_cmds.txt
@@ -0,0 +1,4 @@
+echo "Cash:"
+register cash
+echo "Food:"
+register food


Reviews:
user: login: simonmichael
body: > run can call run 

😂 mind blown!

This sounds great, I look forward to trying, though it might take another day or two. Actually I have no objection to merging, if we can hear from another reviewer/tester.
state: COMMENTED
author_association: OWNER
submitted_at: 2025-02-25T04:07:11Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-02-26T04:27:21Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-02-26T04:28:18Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-02-26T06:40:58Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-26T10:52:38Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-26T10:54:14Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-26T10:57:25Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-26T10:59:37Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-26T19:15:31Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-05T02:17:08Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-05T02:20:38Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T08:54:25Z
user: login: simonmichael
body: The --version, --man, --info, and --tldr flags, with a command, all seem to be ignored. Normally they would show that information instead of running the command's normal action.
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-05T11:04:30Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-05T21:07:23Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T15:31:50Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T16:16:33Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T16:31:44Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T17:32:09Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-06T18:49:20Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-06T18:49:54Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-06T18:53:51Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-06T18:56:55Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T19:05:10Z
user: login: adept
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-06T19:26:10Z
user: login: simonmichael
state: COMMENTED
author_association: OWNER
submitted_at: 2025-03-06T21:21:42Z

Review comments:
diff_hunk: @@ -33,7 +33,7 @@ You can use `#!/usr/bin/env hledger run` in the first line of the file to make i
 
 - If you meant to provide file name as an argument, but made a mistake and a gave file name that does not exist, "run" will attempt to interpret it as a command.
 
-- Numeric flags like `-3` do not work, use long form `--depth 3`
+- You can pass `-f` to the `run` itself, and also to any commands given after it (or in the command file, or via REPL). When specific command does not have `-f` in its flags, it will use the journal(s) specified in the arguments of `run`. If command does have `-f` flag, this journal would be read and its contents would be cache, so if several commands specify the same `-f` flag, they will read the journal only once.
path: hledger/Hledger/Cli/Commands/Run.md
user: login: simonmichael
body: Let's see if this gets confusing or not.

I am reminded of GHCI's `:l(oad) FILE` and `:m(odule) (+-*)MODULE` commands, which let you bring things in and out of memory and move things in and out of scope.
created_at: 2025-02-25T04:04:39Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: splitReaderPrefix should be used here in case it's a prefixed path like `csv:-`.
created_at: 2025-02-26T04:07:27Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: I guess this works by running newMVar only the first time journalCache is called, relying on GHC's usual unsafePerformIO behaviour. 

Is this a common idiom ? Is it behaviour we could rely on with future GHC, or GHC compiling to wasm, or other haskell compilers like microhs ?

Are there any implications for thread safety of the Hledger.Cli library ?
created_at: 2025-02-26T04:11:48Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
+
+addJournalToCache :: Journal -> [String] -> IO ()
+addJournalToCache j journalpaths = modifyMVar_ journalCache $ \cache ->
+  if hasStdin $ dbg1 "addJournalToCache" journalpaths
+  then return cache
+  else return $ Map.insert journalpaths j cache
+
+withJournalCached :: CliOpts -> (Journal -> IO ()) -> IO ()
+withJournalCached cliopts f = do
+  journalpaths <- journalFilePathFromOptsNoDefault cliopts
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: I am a little confused, doesn't this return a single file path ? Wouldn't "journalpath" or "p" be clearer ? Or following my usual convention, "mp" (maybe p).
created_at: 2025-02-26T04:16:18Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
+
+addJournalToCache :: Journal -> [String] -> IO ()
+addJournalToCache j journalpaths = modifyMVar_ journalCache $ \cache ->
+  if hasStdin $ dbg1 "addJournalToCache" journalpaths
+  then return cache
+  else return $ Map.insert journalpaths j cache
+
+withJournalCached :: CliOpts -> (Journal -> IO ()) -> IO ()
+withJournalCached cliopts f = do
+  journalpaths <- journalFilePathFromOptsNoDefault cliopts
+  -- if command does not have -f flags, use the same journal that was given to the "run" invocation
+  let key = case journalpaths of
+              Nothing -> defaultJournalKey
+              Just paths -> NE.toList paths
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: It seems to be converted to a (singleton) list here - why is that ?
created_at: 2025-02-26T04:18:33Z
author_association: OWNER
diff_hunk: @@ -44,6 +44,14 @@ Caveats:
 
 -   Numeric flags like -3 do not work, use long form --depth 3
 
+-   You can pass -f to the run itself, and also to any commands given
+    after it (or in the command file, or via REPL). When specific
+    command does not have -f in its flags, it will use the journal(s)
+    specified in the arguments of run. If command does have -f flag,
+    this journal would be read and its contents would be cache, so if
+    several commands specify the same -f flag, they will read the
+    journal only once.
+
path: hledger/Hledger/Cli/Commands/Run.txt
user: login: simonmichael
body: Typo: "cache". 
Also the wording seems to suggest caching is a special thing that happens when command has -f. Perhaps it could be clarified that all input files are cached.
(I sometimes say "input files", since they're not always .journals).

created_at: 2025-02-26T04:27:03Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Would the uglymemo library be of any help here ? We use it for other things, eg regexps. Ignore if this doesn't make sense.
created_at: 2025-02-26T04:28:18Z
author_association: OWNER
diff_hunk: @@ -81,23 +99,30 @@ parseCommand line =
   -- # begins a comment, ignore everything after #
   takeWhile (not. ((Just '#')==) . headMay) $  words' (strip line)
 
-runCommand :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> [String] -> IO ()
-runCommand findBuiltinCommand j cmdline = do
-  dbg1IO "running command" cmdline
-  -- # begins a comment, ignore everything after #
+-- | Take a single command line (from file, or REPL, or "--"-surrounded block of the args), and run it.
+runCommand :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runCommand findBuiltinCommand cmdline = do
+  dbg1IO "runCommand for" cmdline
   case cmdline of
     "echo":args -> putStrLn $ unwords $ args
     cmdname:args ->
       case findBuiltinCommand cmdname of
-      Nothing -> putStrLn $ unwords (cmdname:args)
-      Just (cmdmode,cmdaction) -> do
-        opts <- getHledgerCliOpts' cmdmode args
-        cmdaction opts j
+        Nothing -> putStrLn $ unwords (cmdname:args)
+        Just (cmdmode,cmdaction) -> do
+          -- Allow "run" to call "run"
+          let cmdaction' = if cmdname == "run" then run findBuiltinCommand else cmdaction
+          -- Even though expandArgsAt is done by the Cli.hs, it stops at the first '--', so we need
+          -- to do it here as well to make sure that each command can use @ARGFILEs 
+          args' <- expandArgsAt args
+          dbg1IO "runCommand final args" (cmdname,args')
+          opts <- getHledgerCliOpts' cmdmode args'
+          withJournalCached opts (cmdaction' opts)
     [] -> return ()
 
-runREPL :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> IO ()
-runREPL findBuiltinCommand j = do
-  putStrLn "Enter hledger commands, or 'help' for help."
+-- | Run an interactive REPL.
+runREPL :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> IO ()
+runREPL findBuiltinCommand = do
+  putStrLn "Enter hledger commands, or 'quit' for help."
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Typo: quit for help.
created_at: 2025-02-26T06:40:58Z
author_association: OWNER
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: This is fixed in later commits. Do you want me to squash everything in a single commit, perhaps?
created_at: 2025-02-26T10:52:38Z
author_association: COLLABORATOR
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: MVar is used specifically for thread safety. Uglymemo does exactly this under the hood, btw: https://github.com/postgres/postgres/blob/0e42d31b0b2273c376ce9de946b59d155fac589c/src/backend/optimizer/path/costsize.c#L1171

So I am not making the situation any worse, as uglymemo is already doing all of this :)

I am not using uglymemo directly because it can't memoize `(a -> IO b)` functions, only pure ones.
created_at: 2025-02-26T10:54:14Z
author_association: COLLABORATOR
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
+
+addJournalToCache :: Journal -> [String] -> IO ()
+addJournalToCache j journalpaths = modifyMVar_ journalCache $ \cache ->
+  if hasStdin $ dbg1 "addJournalToCache" journalpaths
+  then return cache
+  else return $ Map.insert journalpaths j cache
+
+withJournalCached :: CliOpts -> (Journal -> IO ()) -> IO ()
+withJournalCached cliopts f = do
+  journalpaths <- journalFilePathFromOptsNoDefault cliopts
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: This returns a list, if there are several `-f` options. I've called it `mbjournalpaths` in later commit
created_at: 2025-02-26T10:57:25Z
author_association: COLLABORATOR
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
+
+addJournalToCache :: Journal -> [String] -> IO ()
+addJournalToCache j journalpaths = modifyMVar_ journalCache $ \cache ->
+  if hasStdin $ dbg1 "addJournalToCache" journalpaths
+  then return cache
+  else return $ Map.insert journalpaths j cache
+
+withJournalCached :: CliOpts -> (Journal -> IO ()) -> IO ()
+withJournalCached cliopts f = do
+  journalpaths <- journalFilePathFromOptsNoDefault cliopts
+  -- if command does not have -f flags, use the same journal that was given to the "run" invocation
+  let key = case journalpaths of
+              Nothing -> defaultJournalKey
+              Just paths -> NE.toList paths
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: when the command passed to run does not have any "-f" flags, we want it to use whatever journal was given to the "run" itself. Rather that recovering cliopts given to run, i decided to introduce a special cache key that represents "journal given to run". 

Now that i think of this, this wold not work properly with nested run invocations ... I will fix this
created_at: 2025-02-26T10:59:36Z
author_association: COLLABORATOR
diff_hunk: @@ -108,5 +115,46 @@ runREPL findBuiltinCommand j = do
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
+        liftIO $ runCommand findBuiltinCommand $ parseCommand input
         loop
+
+{-# NOINLINE journalCache #-}
+journalCache :: MVar (Map.Map [String] Journal)
+journalCache = unsafePerformIO $ newMVar Map.empty
+
+-- | Key used to cache the journal given in the arguments to 'run'.
+defaultJournalKey :: [String]
+defaultJournalKey = ["journal specified in args of run"]
+
+hasStdin :: [String] -> Bool
+hasStdin fps = "-" `elem` fps
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: Noted, I've made this change
created_at: 2025-02-26T19:15:31Z
author_association: COLLABORATOR
diff_hunk: @@ -38,41 +49,70 @@ runmode = hledgerCommandMode
   )
   cligeneralflagsgroups1
   hiddenflags
-  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...]")
+  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...] OR [command1 args... -- command2 args... -- command3 args...]")
 
--- | The fake run command introduced to break circular dependency
-run' :: CliOpts -> Journal -> IO ()
-run' _opts _j = return ()
+replmode = hledgerCommandMode
+  $(embedFileRelative "Hledger/Cli/Commands/Repl.txt")
+  (
+  []
+  )
+  cligeneralflagsgroups1
+  hiddenflags
+  ([], Nothing)
+
+-- | The fake run/repl command introduced to break circular dependency.
+-- This module needs access to `findBuiltinCommand`, which is defined in Hledger.Cli.Commands
+-- However, Hledger.Cli.Commands imports this module, which creates circular dependency.
+-- We expose this do-nothing function so that it could be included in the list of all commands inside
+-- Hledger.Cli.Commands and ensure that "run" is recognized as a valid command by the Hledger.Cli top-level
+-- command line parser. That parser, however, would not call run'. It has a special case for "run", and
+-- will call "run" (see below), passing it `findBuiltinCommand`, thus breaking circular dependency.
+runOrReplStub :: CliOpts -> Journal -> IO ()
+runOrReplStub _opts _j = return ()
+
+-- | Default input files + InputOpts that would be used by commands if
+--   there is no explicit alternative given
+newtype DefaultRunJournal = DefaultRunJournal (NE.NonEmpty String)
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Not a release blocker, but I think we'll be able to make this clearer. 
It's not a journal, it's a list of input file paths.
What does "+ InputOpts" mean here ? I thought maybe it meant "also includes options like --infer-equity" but it seems not.
created_at: 2025-03-05T02:17:08Z
author_association: OWNER
diff_hunk: @@ -38,41 +49,70 @@ runmode = hledgerCommandMode
   )
   cligeneralflagsgroups1
   hiddenflags
-  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...]")
+  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...] OR [command1 args... -- command2 args... -- command3 args...]")
 
--- | The fake run command introduced to break circular dependency
-run' :: CliOpts -> Journal -> IO ()
-run' _opts _j = return ()
+replmode = hledgerCommandMode
+  $(embedFileRelative "Hledger/Cli/Commands/Repl.txt")
+  (
+  []
+  )
+  cligeneralflagsgroups1
+  hiddenflags
+  ([], Nothing)
+
+-- | The fake run/repl command introduced to break circular dependency.
+-- This module needs access to `findBuiltinCommand`, which is defined in Hledger.Cli.Commands
+-- However, Hledger.Cli.Commands imports this module, which creates circular dependency.
+-- We expose this do-nothing function so that it could be included in the list of all commands inside
+-- Hledger.Cli.Commands and ensure that "run" is recognized as a valid command by the Hledger.Cli top-level
+-- command line parser. That parser, however, would not call run'. It has a special case for "run", and
+-- will call "run" (see below), passing it `findBuiltinCommand`, thus breaking circular dependency.
+runOrReplStub :: CliOpts -> Journal -> IO ()
+runOrReplStub _opts _j = return ()
+
+-- | Default input files + InputOpts that would be used by commands if
+--   there is no explicit alternative given
+newtype DefaultRunJournal = DefaultRunJournal (NE.NonEmpty String)
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: PS I found it useful to derive Show here for debugging, might be worth adding.
created_at: 2025-03-05T02:19:09Z
author_association: OWNER
diff_hunk: @@ -38,41 +49,70 @@ runmode = hledgerCommandMode
   )
   cligeneralflagsgroups1
   hiddenflags
-  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...]")
+  ([], Just $ argsFlag "[COMMANDS_FILE1 COMMANDS_FILE2 ...] OR [command1 args... -- command2 args... -- command3 args...]")
 
--- | The fake run command introduced to break circular dependency
-run' :: CliOpts -> Journal -> IO ()
-run' _opts _j = return ()
+replmode = hledgerCommandMode
+  $(embedFileRelative "Hledger/Cli/Commands/Repl.txt")
+  (
+  []
+  )
+  cligeneralflagsgroups1
+  hiddenflags
+  ([], Nothing)
+
+-- | The fake run/repl command introduced to break circular dependency.
+-- This module needs access to `findBuiltinCommand`, which is defined in Hledger.Cli.Commands
+-- However, Hledger.Cli.Commands imports this module, which creates circular dependency.
+-- We expose this do-nothing function so that it could be included in the list of all commands inside
+-- Hledger.Cli.Commands and ensure that "run" is recognized as a valid command by the Hledger.Cli top-level
+-- command line parser. That parser, however, would not call run'. It has a special case for "run", and
+-- will call "run" (see below), passing it `findBuiltinCommand`, thus breaking circular dependency.
+runOrReplStub :: CliOpts -> Journal -> IO ()
+runOrReplStub _opts _j = return ()
+
+-- | Default input files + InputOpts that would be used by commands if
+--   there is no explicit alternative given
+newtype DefaultRunJournal = DefaultRunJournal (NE.NonEmpty String)
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: This is an obsolete comment, sorry :( I'll clean this up
created_at: 2025-03-05T08:54:25Z
author_association: COLLABORATOR
diff_hunk: @@ -81,32 +121,92 @@ parseCommand line =
   -- # begins a comment, ignore everything after #
   takeWhile (not. ((Just '#')==) . headMay) $  words' (strip line)
 
-runCommand :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> [String] -> IO ()
-runCommand findBuiltinCommand j cmdline = do
-  dbg1IO "running command" cmdline
-  -- # begins a comment, ignore everything after #
+-- | Take a single command line (from file, or REPL, or "--"-surrounded block of the args), and run it.
+runCommand :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runCommand defaultJournalOverride findBuiltinCommand cmdline = do
+  dbg1IO "runCommand for" cmdline
   case cmdline of
     "echo":args -> putStrLn $ unwords $ args
     cmdname:args ->
       case findBuiltinCommand cmdname of
-      Nothing -> putStrLn $ unwords (cmdname:args)
-      Just (cmdmode,cmdaction) -> do
-        opts <- getHledgerCliOpts' cmdmode args
-        cmdaction opts j
+        Nothing -> error' $ "Unrecognized command: " ++ unwords (cmdname:args)
+        Just (cmdmode,cmdaction) -> do
+          -- Even though expandArgsAt is done by the Cli.hs, it stops at the first '--', so we need
+          -- to do it here as well to make sure that each command can use @ARGFILEs
+          args' <- replaceNumericFlags <$> expandArgsAt args
+          dbg1IO "runCommand final args" (cmdname,args')
+          opts <- getHledgerCliOpts' cmdmode args'
+          withJournalCached (Just defaultJournalOverride) opts $ \(j,key) -> do
+            if cmdname == "run" -- allow "run" to call "run"
+              then run (Just key) findBuiltinCommand opts
+              else cmdaction opts j
     [] -> return ()
 
-runREPL :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> IO ()
-runREPL findBuiltinCommand j = do
-  putStrLn "Enter hledger commands, or 'help' for help."
-  runInputT defaultSettings loop
+-- | Run an interactive REPL.
+runREPL :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> IO ()
+runREPL defaultJournalOverride findBuiltinCommand = do
+  isTerminal <- hIsTerminalDevice stdin
+  if not isTerminal
+    then runInputT defaultSettings (loop "")
+    else do
+      putStrLn "Enter hledger commands. To exit, enter 'quit' or 'exit', or send EOF."
+      runInputT defaultSettings (loop "% ")
   where
-  loop :: InputT IO ()
-  loop = do
-    minput <- getInputLine "% "
+  loop :: String -> InputT IO ()
+  loop prompt = do
+    minput <- getInputLine prompt
     case minput of
       Nothing -> return ()
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
-        loop
+        liftIO $ (runCommand defaultJournalOverride findBuiltinCommand $ parseCommand input)
+                  `catch` (\(e::ErrorCall) -> putStr $ show e)
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: ```haskell
                  `catches` 
                  [Handler (\(e::ErrorCall) -> putStr $ show e)
                  ,Handler (\(_::ExitCode) -> return ())
                  ,Handler (\UserInterrupt -> return ())
                  ]
```

This also catches 

- program exit after `test`
- program exit after `CMD --help` (getHledgerCliOpts' calls exitSuccess)
- the first control-C.

The last is questionable; unfortunately I haven't found a way to catch the second control-C, other than using the non-portable System.Posix.Signals API. (Seems to be hard-coded RTS behaviour.)

created_at: 2025-03-05T10:59:56Z
author_association: OWNER
diff_hunk: @@ -81,32 +121,92 @@ parseCommand line =
   -- # begins a comment, ignore everything after #
   takeWhile (not. ((Just '#')==) . headMay) $  words' (strip line)
 
-runCommand :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> [String] -> IO ()
-runCommand findBuiltinCommand j cmdline = do
-  dbg1IO "running command" cmdline
-  -- # begins a comment, ignore everything after #
+-- | Take a single command line (from file, or REPL, or "--"-surrounded block of the args), and run it.
+runCommand :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> [String] -> IO ()
+runCommand defaultJournalOverride findBuiltinCommand cmdline = do
+  dbg1IO "runCommand for" cmdline
   case cmdline of
     "echo":args -> putStrLn $ unwords $ args
     cmdname:args ->
       case findBuiltinCommand cmdname of
-      Nothing -> putStrLn $ unwords (cmdname:args)
-      Just (cmdmode,cmdaction) -> do
-        opts <- getHledgerCliOpts' cmdmode args
-        cmdaction opts j
+        Nothing -> error' $ "Unrecognized command: " ++ unwords (cmdname:args)
+        Just (cmdmode,cmdaction) -> do
+          -- Even though expandArgsAt is done by the Cli.hs, it stops at the first '--', so we need
+          -- to do it here as well to make sure that each command can use @ARGFILEs
+          args' <- replaceNumericFlags <$> expandArgsAt args
+          dbg1IO "runCommand final args" (cmdname,args')
+          opts <- getHledgerCliOpts' cmdmode args'
+          withJournalCached (Just defaultJournalOverride) opts $ \(j,key) -> do
+            if cmdname == "run" -- allow "run" to call "run"
+              then run (Just key) findBuiltinCommand opts
+              else cmdaction opts j
     [] -> return ()
 
-runREPL :: (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> Journal -> IO ()
-runREPL findBuiltinCommand j = do
-  putStrLn "Enter hledger commands, or 'help' for help."
-  runInputT defaultSettings loop
+-- | Run an interactive REPL.
+runREPL :: DefaultRunJournal -> (String -> Maybe (Mode RawOpts, CliOpts -> Journal -> IO ())) -> IO ()
+runREPL defaultJournalOverride findBuiltinCommand = do
+  isTerminal <- hIsTerminalDevice stdin
+  if not isTerminal
+    then runInputT defaultSettings (loop "")
+    else do
+      putStrLn "Enter hledger commands. To exit, enter 'quit' or 'exit', or send EOF."
+      runInputT defaultSettings (loop "% ")
   where
-  loop :: InputT IO ()
-  loop = do
-    minput <- getInputLine "% "
+  loop :: String -> InputT IO ()
+  loop prompt = do
+    minput <- getInputLine prompt
     case minput of
       Nothing -> return ()
       Just "quit" -> return ()
       Just "exit" -> return ()
       Just input -> do
-        liftIO $ runCommand findBuiltinCommand j $ parseCommand input
-        loop
+        liftIO $ (runCommand defaultJournalOverride findBuiltinCommand $ parseCommand input)
+                  `catch` (\(e::ErrorCall) -> putStr $ show e)
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: Thanks, added!
created_at: 2025-03-05T21:07:23Z
author_association: COLLABORATOR
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Great! It was there in plain sight...

Would a call to uglymemo be enough here ?
created_at: 2025-03-06T14:54:25Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: I note my earlier run.test still fails (except for one test, for which I've just pushed a small fix to master).
created_at: 2025-03-06T15:18:27Z
author_association: OWNER
diff_hunk: @@ -0,0 +1,136 @@
+# * run command
+
+# ** 0. Run one command from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- register cash
+>
+2017-01-01 groceries            assets:cash                  $-100         $-100
+
+
+# ** 0. Run two commands from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- echo "Cash:" -- register cash -- echo 'Food:' -- register food
+>
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+
+# ** 0. Run two commands from command line with different input options
+<
+2017-01-01 investment
+   assets:cash  -$100
+   expenses:food
+
+~ 2017-02-01  next month groceries
+   assets:cash  -$120
+   expenses:food
+$ hledger run -f- -- register cash -- echo '=====' -- register --forecast food
+>
+2017-01-01 investment           assets:cash                  $-100         $-100
+=====
+2017-01-01 investment           expenses:food                 $100          $100
+2017-02-01 next month grocer..  expenses:food                 $120          $220
+
+# ** 0. Run commands from file
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- run_cmds.txt
path: hledger/test/run.test
user: login: simonmichael
body: Seems like a mistake here ? Is the test supposed to be reading from both stdin and run_cmds.txt ?
created_at: 2025-03-06T15:27:16Z
author_association: OWNER
diff_hunk: @@ -0,0 +1,136 @@
+# * run command
+
+# ** 0. Run one command from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- register cash
+>
+2017-01-01 groceries            assets:cash                  $-100         $-100
+
+
+# ** 0. Run two commands from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- echo "Cash:" -- register cash -- echo 'Food:' -- register food
+>
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+
+# ** 0. Run two commands from command line with different input options
+<
+2017-01-01 investment
+   assets:cash  -$100
+   expenses:food
+
+~ 2017-02-01  next month groceries
+   assets:cash  -$120
+   expenses:food
+$ hledger run -f- -- register cash -- echo '=====' -- register --forecast food
+>
+2017-01-01 investment           assets:cash                  $-100         $-100
+=====
+2017-01-01 investment           expenses:food                 $100          $100
+2017-02-01 next month grocer..  expenses:food                 $120          $220
+
+# ** 0. Run commands from file
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- run_cmds.txt
path: hledger/test/run.test
user: login: simonmichael
body: Actually I guess it's right, I was confused by `hledger run -f FILE1 FILE2`. This reads data from FILE1 and commands from FILE2.
created_at: 2025-03-06T15:29:36Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Those tests are failing because the options before `run` get moved to the end by moveFlagsAfterCommand.
```
-- - XXX Relative order of flags is mostly but not entirely preserved, currently:
--   pre-command flags get moved to the end, after post-command flags. 
```
created_at: 2025-03-06T16:16:33Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: (You said this yesterday, I understand it now.)
created_at: 2025-03-06T16:31:44Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: I have pushed a fix to master; now only two tests fail:

```
# ** 0. Run with empty command line, does nothing.
$ hledger -f- run
```
This one gives "illegal operation (handle is closed)".

```
# ** 0. Run two commands from command line, no leading --
$ hledger -f- run files -- files
>
-
-
```
This one runs `hledger files files`, printing nothing.
Something (cmdargs ?) consumes the first --. If we can't avoid that, we could maybe have `run` raise an error if the first non-file argument is not -- ?
created_at: 2025-03-06T17:32:09Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: It is either cmdargs or Cli.hs. Whatever this is, the end result is as follow:

- Commandline "hledger -f- run files -- files" leads to Run.hs getting "files files" as args
- Commandline "hledger -f- run -- files -- files" leads to Run.hs getting "files -- files" as args

So Run.hs does not have enough info to know if there was leading "--" or not :(
created_at: 2025-03-06T18:49:20Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,136 @@
+# * run command
+
+# ** 0. Run one command from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- register cash
+>
+2017-01-01 groceries            assets:cash                  $-100         $-100
+
+
+# ** 0. Run two commands from command line
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- -- echo "Cash:" -- register cash -- echo 'Food:' -- register food
+>
+Cash:
+2017-01-01 groceries            assets:cash                  $-100         $-100
+Food:
+2017-01-01 groceries            expenses:food                 $100          $100
+
+# ** 0. Run two commands from command line with different input options
+<
+2017-01-01 investment
+   assets:cash  -$100
+   expenses:food
+
+~ 2017-02-01  next month groceries
+   assets:cash  -$120
+   expenses:food
+$ hledger run -f- -- register cash -- echo '=====' -- register --forecast food
+>
+2017-01-01 investment           assets:cash                  $-100         $-100
+=====
+2017-01-01 investment           expenses:food                 $100          $100
+2017-02-01 next month grocer..  expenses:food                 $120          $220
+
+# ** 0. Run commands from file
+<
+2017-01-01 groceries
+   assets:cash  -$100
+   expenses:food
+$ hledger run -f- run_cmds.txt
path: hledger/test/run.test
user: login: adept
body: Yes, exactly right
created_at: 2025-03-06T18:49:54Z
author_association: COLLABORATOR
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: Re uglymemo: it only memoizes (a->b) functions, it can't memoize (a-> IO b) (dont really understand why).

As noted previously, this code is conceptually the same as internals of uglumemo, so it is not more ugly that it is (but no less ugly either :)
created_at: 2025-03-06T18:53:51Z
author_association: COLLABORATOR
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: https://hackage.haskell.org/package/cmdargs-0.10.22/docs/System-Console-CmdArgs-Explicit.html says "Parsing rules: Command lines are parsed as per most GNU programs. Short arguments single letter flags start with -, longer flags start with --, and everything else is considered an argument. Anything after -- alone is considered to be an argument. For example:"

So I am pretty sure that it is cmdargs eating first "--"

EDIT: yes, it does
created_at: 2025-03-06T18:56:54Z
author_association: COLLABORATOR
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: Insert another -- at some point ? :)
created_at: 2025-03-06T19:05:10Z
author_association: OWNER
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: adept
body: Well... If we are seriously considering this, maybe getting the original command line or if getArgs and looking at it could get us somewhere, but this feels... horrible :) 
created_at: 2025-03-06T19:26:10Z
author_association: COLLABORATOR
diff_hunk: @@ -211,11 +214,25 @@ withJournalCached defaultJournalOverride cliopts cmd = do
             return (cache, journal)
           Nothing -> do
             dbg1IO "readAndCacheJournalFile reading and caching journals" (fp, iopts)
-            journal <- runExceptT $ readJournalFile iopts fp
+            journal <- runExceptT $ if snd (splitReaderPrefix fp) == "-" then readStdin else readJournalFile iopts fp
             either error' (\j -> return (Map.insert (ioptsWithoutReportSpan,fp) j cache, j)) journal
       where
         -- InputOptions contain reportspan_ that is used to calculare forecast period,
         -- that is used by journalFinalise to insert forecast transactions.addHeaderBorders
         -- For the purposes of caching, we want to ignore this, as it is only used for forecast
         -- and it is sufficient to include forecast_ in the InputOptions that we use as a key.
-        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan } 
+        ioptsWithoutReportSpan = iopts { reportspan_ = emptydatespan }
+        -- Read stdin, or if we read it alread, use a cache
+        -- readStdin :: InputOpts -> ExceptT String IO Journal
+        readStdin = do
+          stdinContent <- liftIO $ modifyMVar stdinCache $ \cache ->
+            case cache of
+              Just cached -> do
+                dbg1IO "readStdin using cached stdin" "-"
+                return (cache, cached)
+              Nothing -> do
+                dbg1IO "readStdin reading and caching stdin" "-"
+                stdinContent <- readFileOrStdinPortably "-"
+                return (Just stdinContent, stdinContent)
+          hndl <- liftIO $ inputToHandle stdinContent
+          readJournal iopts Nothing hndl
path: hledger/Hledger/Cli/Commands/Run.hs
user: login: simonmichael
body: FTR: we did that, excellent workaround.
created_at: 2025-03-06T21:21:42Z
author_association: OWNER

PR comments:
user: login: simonmichael
created_at: 2025-02-25T04:08:09Z
body: Actually, please enlighten us: what happens with run in run ?
author_association: OWNER
user: login: adept
created_at: 2025-02-25T18:45:07Z
body: Lets run (pun intended!) through some examples:

You do: `hledger run -f file1.journal -- cmd1 -- cmd2 -- cmd3` (here and below `cmd1` etc refers to some hledger command)
What happens: `run` loads `file1.journal` and runs three commands on that journal

You do: `hledger run -- cmd1 -- cmd2 -- cmd3`
What happens: `run` loads your default journal and runs three commands on that journal

You do: `hledger run -f file1.journal -- cmd1 -- cmd2 -f file2.journal -- cmd3 -- cmd4 -f file2.journal`
What happens: `run` loads `file1.journal` and runs `cmd1` on it. It then loads `flie2.journal` and runs `cmd2` on it. It then runs `cmd3` on the cached contents of `file1.journal` and finally runs `cmd4` on a cached contents of `file2.journal`

You do: `hledger run -- cmd1 -- run -- cmd2`, trying to nest `run` invocations on the command line
What happens: `run` loads your default journal, runs `cmd1` on it, and then drops you into REPL. 
We can't effectively "nest" `--`-separated command sections on the command line, so if you specify everything on the command line, nested `run` invocations are not really possible

You do: `hledger run -f file1.journal script1.txt`, where script1.txt contains:
```
cmd1
run -f file2.journal script2.txt
cmd2 -f file2.journal
```
What happens: `run` will load `file1.journal`, execute `cmd1` on it, then it will load (and cache) `file2.journal` and run commands from `script2.txt` on it, then it will use cached `file2.journal` to run `cmd2`

Does it clear things up a little bit, or makes them worse? :)
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-02-26T03:06:39Z
body: That is very helpful, and it probably deserves to be in the manual, as long as things work this way. 

Though, there's a risk of scaring users with complexity which they hopefully won't encounter. (But they probably will.)

Some questions:

> You do: hledger run -f file1.journal -- cmd1 -- cmd2 -f file2.journal -- cmd3 -- cmd4 -f file2.journal
...
> It then runs cmd3 on the cached contents of file1.journal and finally runs cmd4 on a cached contents of file2.journal

I find that intuitive for a REPL but surprising on the command line, where (at least for hledger) right-most options usually take precedence. So this example will be helpful.

> You do: hledger run -- cmd1 -- run -- cmd2, trying to nest run invocations on the command line
> What happens: run loads your default journal, runs cmd1 on it, and then drops you into REPL.

And cmd2 would never be executed ? Or would it execute if you exit the REPL without error ? Let's spell that out for completeness.


author_association: OWNER
user: login: simonmichael
created_at: 2025-02-26T03:13:48Z
body: Related: 

I gave the claude code and aider REPLs a test drive today. They are very pretty, and give ideas for what could be added to a hledger REPL.
I almost wish the REPL front end could be written in a "softer" language like python, which could evolve faster and use python's polished TUI libs. But I guess that's out, because it would complicate packaging and it would lose the key benefit of having native hledger data in memory.
author_association: OWNER
user: login: adept
created_at: 2025-02-26T19:17:25Z
body: @simonmichael I realize that there is quite a lot of commits here, and I am always unsure if this is a good thing or not.

I am hesitant to squish them, at the same time I feel that they maybe make review more complicated? (Also, I dont use git often, so I am always scared of history rewriting lest I mess something up :)

My reasoning at the moment is that one could always look at "Files changed" tab to see the overall diff, and at individual commits if they want to ...
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-02-26T21:01:16Z
body: @adept I'm fine with leaving the commits as is for now if it's easier, though also feel free to rewrite them into fewer cleaner ones and force-push if you want. I can view them both ways as you say. At merge time we can decide separately how best to merge.
author_association: OWNER
user: login: adept
created_at: 2025-02-27T20:31:17Z
body: todo:
- DONE: split run and repl
- DONE: fix small issue with nested "run"s
- reconsider "echo"
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-02-28T00:19:04Z
body: Noting some things we've discussed in chat, for the record:

I'm liking the split; at the very least it provides intuitive command names for both the batch mode and the interactive mode, and I think users will find it easier to grok these one at a time. Having said that, the split is not perfect:

- `run` can no longer read commands from stdin; I think it should support that.

- `repl`, in addition to its interactive mode, can read commands from stdin and process them in batch mode. This will be counterintuitive to some, though not to unix hackers. Ideally in this mode it would not print the % prompts. Possibly it could remain at the prompt after those commands complete (might as well do something `run` won't do).

But it's early days. At the moment run and repl are still so similar it's not yet clear if they are best as one command or as separate commands. I do know as one command it was a lot to explain, test, and for a new user to learn (see #2338 run: UX notes). But then again these are power user commands.

We'll keep the echo command for now and remove the "display unrecognized commands as output" fallback for robustness.

A few typos need fixing ("LEDGER_JOURNAL", "agument", "anoter") and in the commands list, "repl" needs a one space indent.

author_association: OWNER
user: login: simonmichael
created_at: 2025-02-28T00:22:28Z
body: > But then again these are power user commands.

Counterargument: 
- combining commands into custom report scripts will be a pretty common use case
- being able to run reports faster with less typing (and in future, benefit from more UI aids) will also be quite attractive to new users

author_association: OWNER
user: login: adept
created_at: 2025-02-28T12:23:02Z
body: More changes:
- run can again read from stdin
- repl (and run) hides prompt and start banner when used non-interactively
- repl does not stop on errors
- typos fixed
- run and repl complain on unrecognized input instead of printing it out
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-01T21:33:56Z
body: @simonmichael by my count, there is nothing else planned (so far) left to implement. (just wanted to explicitly say that I am not sitting on any in-progress or uncommitted changes here :)
author_association: COLLABORATOR
user: login: Xitian9
created_at: 2025-03-02T11:31:30Z
body: I've played around with this a bit to see how it works. Thanks for the work!

The main problem I see is that `run` does not work with `--forecast` in internal commands. I suspect this has something to do with when journal finalisation occurs, and how that differs among different subcommands.

```
$ cat test.journal
2025-01-01  Buy apple pie
    expenses:food              $20
    assets:cash               -$20

~ every 6 months from 2025-02-01  Buy soylent green
    expenses:food:not people   $10
    assets:cash               -$10

$ cat cmdtest.txt
register --forecast

# It works if you call register with --forecast
$ hledger -f test.journal register --forecast
2025-01-01 Buy apple pie                  expenses:food                            $20           $20
                                          assets:cash                             $-20             0
2025-02-01 Buy soylent green              expenses:food:not people                 $10           $10
                                          assets:cash                             $-10             0
2025-08-01 Buy soylent green              expenses:food:not people                 $10           $10
                                          assets:cash                             $-10             0

# It works if you supply --forecast on the command line
$ hledger -f test.journal --forecast run cmdtest.txt
2025-01-01 Buy apple pie                  expenses:food                            $20           $20
                                          assets:cash                             $-20             0
2025-02-01 Buy soylent green              expenses:food:not people                 $10           $10
                                          assets:cash                             $-10             0
2025-08-01 Buy soylent green              expenses:food:not people                 $10           $10
                                          assets:cash                             $-10             0

# It does not work if --forecast is only in the run command
$ hledger -f test.journal run cmdtest.txt
2025-01-01 Buy apple pie                  expenses:food                            $20           $20
                                          assets:cash                             $-20             0
```

I suspect this may affect other journal modification commands (auto postings, aliases), but haven't explored in depth.

Here are some other minor things that came up when I attempted to use it. None are blockers. 

- `hledger run` does not pass common arguments to its subcommands, for example `hledger run --pretty=yes` doesn't pass `--pretty=yes` to all subcommands. Whether it should is a good question, but I naively thought it might.
- `hledger run` does not pass to a pager the way other commands do. Again, whether it should is an open question.
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-03T00:16:08Z
body: Regarding forecast:
- finalisation is done in the parser
- files are cached by the name (only)

Also internal commands, when not given -f, just reuse the journal that was passed to run, bypassing cache entirely. 

"Cache invalidation, naming, and off by one errors - two biggest issues in programming" :)

I can see how this is going to be an issue with -b -e probably as well, right?

Back to the drawing board... 
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-03T20:07:27Z
body: Ok, let's write down some words.

This is how I want `run` to behave:

1. `run`, like any other command, would either read a default journal, or a journal given by `-f` , This should take into account other command-line flags that influence journal reading/parsing.

2. This journal is then passed to all the commands that `run` runs, except:
   - If the command is explicitly given `-f`, it will operate on that input file (+ whatever other command line options were given)
   - If the command is explicitly given some options that are influencing input, but no `-f`, then it would read the same input files as `run` read (implicitly or explicitly), with those options applied

3. As a consequence of 1+2, if run was was given `-f non-default.journal`, then the only way to read a default journal for some command is to give it `-f default.journal`. Default journal would never be read implicitly in this case.

4. For nested run calls, we first determin which journal it should operate on using rules (1),(2),(3), and then we start it and all the rules above apply to its behavior

Can we somehow get rid of implicit behavior of (2)? 

We could require `-f` to be always given to any command. This is ugly and bad ui

We could introduce some special pseudo-name so that one can say  (semantically) `-f <the same>`. This is also ugly, and special syntax is bad.

We can introduce scopes (gah!), like:
```
with -f some.journal --forecast:
   do this
   and this
   and this
```
and commands inside `with` blocks are forbidden to have `-f`. This looks like "even advanced system contain 50% of the implementation of lisp", and if we go there, we might as well think about embedding lua or python :)
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-03T20:18:05Z
body: Now, lets figure out what influences ingestion of input files.

Commands (other than `run`) dont read input files, `Cli.hs` does this for them. It does this via `withJournalDo`, which takes `CliOpts`. This is how that call unfolds:

`withJournalDo cliopts` calls:
- `readJournalFiles inputopts`
    - lots of functions that eventually call `readJournal iopts`, which in turn:
       -  determines reader from `iopts`
       -  calls `reader.readFn iopts`, where `readFn` is somethng like `JournalReader.reader` ....
- `journalTransform cliopts`

Ok, so iopts are essentially "part of the journal" (as they are passed into parser).
I think i will modify the code to cache files by (iopts, filename), and to apply `journalTransform` to the cached value every time journal is requested .. I think this should definitely fix the `--forecast` bug @Xitian9 experiened, and maybe fix all other issues as well.

author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-03T20:56:34Z
body: @Xitian9 I've pushed this change, your forecast example now works :)
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-03-03T21:30:49Z
body: I appreciate the valuable additional review/discussion/updates here, hope to catch up soon.
author_association: OWNER
user: login: Xitian9
created_at: 2025-03-04T23:55:13Z
body: Thanks! I've run it and can confirm that the `--forecast` example now works. However, speed seems to have taken a big hit, and running my example (5 balance commands in a single run script) is no faster than running hledger 5 separate times.
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-04T23:58:20Z
body: Hmm. Do you mind running it with --debug 1 and looking at whether it caches files or reads them every time? 
author_association: COLLABORATOR
user: login: Xitian9
created_at: 2025-03-05T00:09:22Z
body: It seems to be reading and caching 5 times, and using the cache once. With 5 subcommands, that would be 1 attempt for each subcommand, and 1 for the overall run command. There is only one successful cache hit, probably due to the fact that the reportspan and forecast options change for most of them.
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-05T00:57:24Z
body: Would you be able to share the options that you use for all these commands? 
author_association: COLLABORATOR
user: login: Xitian9
created_at: 2025-03-05T01:08:07Z
body: ```
balance      --pretty=yes --value=end --daily -b "13 days ago" -e tomorrow @expenses-report.args                                  --output-file=/tmp/tmp.V86rCPRgsp/expenses.out
balancesheet --pretty=yes --tree --layout=wide @netliquid.args                                                                    --output-file=/tmp/tmp.V86rCPRgsp/netliquid.out
balance      --pretty=yes --tree @pay.args --period "every 21st day of month from 2025-02-21 to 2025-03-21"                       --output-file=/tmp/tmp.V86rCPRgsp/paynow.out
balance      --pretty=yes --tree @pay.args --period "every 21st day of month from 2025-02-21 to 2025-03-21" --forecast            --output-file=/tmp/tmp.V86rCPRgsp/paymonth.out
balance      --pretty=yes --tree @pay.args --period "every 21st day of month from 2025-02-21 to 2025-05-21" --forecast receivable --output-file=/tmp/tmp.V86rCPRgsp/pay3month.out
```

The arg files contain regular expression account matching and depth arguments.

It's the `balancesheet` command that results in the successful cache lookup, probably because it has no `begin` and `end`, meaning it would match the cache made by the overall run command.
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-03-05T02:05:11Z
body: I'm caught up now. I see how this is complicated. And yes, the overall processing pipeline may have grown a few more offshoots than it should have, perhaps some cleanup will help later. 

I know users will have "fun" figuring out the effects/scopes also. But I suppose that optimal [functional spec](https://github.com/simonmichael/hledger/pull/2328#issuecomment-2695425992) that adept wrote will be the most intuitive overall, once understood, since it "follows the rules". If we can achieve it. But if we must impose some restrictions / document some limitations for now, that's ok too. 
author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T02:07:45Z
body: For the record I'm overdue for the ideal release date, which is my fault, but that is flexible so it's ok to spend more time tinkering with this if needed. In fact I think it's good that we're doing that. This is a big feature and the most prominent in 1.42 probably.
author_association: OWNER
user: login: adept
created_at: 2025-03-05T08:56:13Z
body: @Xitian9 I figured out why, it is because of this guy sneakily living inside InputOpts, so that `journalFinalise` would be able to add forecast transactions:
```
    ,reportspan_        :: DateSpan             -- ^ a dirty hack keeping the query dates in InputOpts. This rightfully lives in ReportSpec, but is duplicated here.
```

I'll push a thing to work around this
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-05T09:04:45Z
body: > For the record I'm overdue for the ideal release date, which is my fault, but that is flexible so it's ok to spend more time tinkering with this if needed. In fact I think it's good that we're doing that. This is a big feature and the most prominent in 1.42 probably.

I feel that we are at the final stage of ironing the wrinkles out, it should not be long :)
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-05T09:05:30Z
body: @Xitian9 if you can give it another spin now it would be great. It seem to be behaving better in my tests
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-03-05T19:21:07Z
body: `run`: when specifying commands on the command line, the requirement for a leading -- is not mentioned in the command line help and without it the results can be confusing, eg try 
```
hledger run files
hledger run files -- files
hledger run files -- files -- files
hledger run files -- files -- files -- files
```

author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T19:24:16Z
body: Actually I see some confusing results even with a leading `--`. Eg shouldn't this print `-` twice ? It prints `-` once and $LEDGER_FILE once.
```
echo | hledger -f- run -- files -- files
```
author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T19:36:56Z
body: Here's a mockup of `run.test`:
```
# * run command

# ** 0. Run with empty command line, does nothing.
$ hledger -f- run

# ** 0. Run one command from command line, no leading --
$ hledger -f- run files
>
-

# ** 0. Run one command from command line, with leading --
$ hledger -f- run -- files
>
-

# ** 0. Run two commands from command line, no leading --
$ hledger -f- run files -- files
>
-
-

# ** 0. Run two commands from command line, with leading --
$ hledger -f- run -- files -- files
>
-
-
```

author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T20:30:19Z
body: Adding Repl.txt to package.yaml extra-source-files will make `just test` pass.
author_association: OWNER
user: login: adept
created_at: 2025-03-05T21:12:28Z
body: > Actually I see some confusing results even with a leading `--`. Eg shouldn't this print `-` twice ? It prints `-` once and $LEDGER_FILE once.
> 
> ```
> echo | hledger -f- run -- files -- files
> ```

Ugh. This is because `Cli.hs` rearranges arguments even before they get to `run`, and `run` receives this:

```
run -- files -- files -f-
```

Perhaps `moveFlagsAfterCommand` (defined in Cli.hs) should stop at first `--`? I feel this might be important for add-on commands as well, maybe?
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-03-05T21:17:34Z
body: I guess our command line processing isn't complicated enough yet. 😂

This sounds like a sensible sort of thing, if tests pass.
author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T21:53:41Z
body: To fix that failing commands test, I needed to add `commands` to the list at section 6.5.2 in main. These are the commands that don't (and shouldn't) read an input file.

`run` and `repl` currently require an input file, eg `LEDGER_FILE= hledger repl` fails, but they probably shouldn't.

author_association: OWNER
user: login: simonmichael
created_at: 2025-03-05T21:55:15Z
body: Cc'ing here for the record:

> I'm sorry for the complexity of command line processing. It's just trying to make everything work as a user would want/expect... while also working around an annoying limitation/mystery of cmdargs (can't ignore unrecognised things). And doing all the same stuff with dynamically discovered addon commands.

author_association: OWNER
user: login: adept
created_at: 2025-03-05T22:04:05Z
body: I've tried to add `run.test`, but ran into a weird error: when tests run, i get the below:
```
hledger: <stdin>: hGetBuffering: illegal operation (handle is closed)
```

I dont understand where hGetBuffering is coming from :( I suspect it is "is stdin a terminal?" check, but I am not sure :( 
A problem for tomorrow, I think
author_association: COLLABORATOR
user: login: adept
created_at: 2025-03-06T11:57:01Z
body: Ok, I figured the hGetBufferring thing - it was an attempt to read stdin twice ...

I fixed it and added `run.test`
author_association: COLLABORATOR
user: login: simonmichael
created_at: 2025-03-06T21:53:55Z
body: Time to merge this. Thank you!
author_association: OWNER
user: login: Xitian9
created_at: 2025-03-07T21:47:07Z
body: Just following up to confirm here that my example now has 6 calls, of which 4 successfully use the cache, consistent with what is expected if `--forecast` needs to regenerate. Thanks!
author_association: COLLABORATOR