WhereIsAI/github-issue-similarity huggingface dataset:
Issue 1:
I found that sometimes function overwriting does not work right away. The method redefinition warning gets triggered, but the old function definition is used.

Here is a minimal test case which shows the bug (the last call fails, but calling f() again then produces 2 as expected):

f() = 1

# at least two calls are required to trigger the bug
f()
f()

f() = 2

@assert f() == 2 # fails

This is on version 0.5.0-dev+4882.

Issue 2:
if I define a function d2 before a function d1 which calls d2 then change d2, d1 uses the old definition for d2.

I assume this is because it is all precompiled, but maybe there should be a note warning of this? Or would it be possible to replace the old definition with a longjmp to the new one?

(Mostly important for the REPL, since I don't always do a full load)

julia> function d2()
a
end

julia> function d()
d2()
end

julia> d()
in d: a not defined

julia> function d2()
b=2
end

julia> d()
in d: a not defined

julia> d2
Methods for generic function d2
d2() at prompt:2

julia> function d()
d2()
end

julia> d()
2
Issue 1:
Challenge create-a-media-query has an issue.

User Agent is: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0.

i get the following error message: "Your p tag should have the font-size of 10px when the device height is smaller than 800px."

<style>
@media (max-height: 800px) {
p {font-size: 10px;}
}
</style>

<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus quis tempus massa. Aenean erat nisl, gravida vel vestibulum cursus, interdum sit amet lectus. Sed sit amet quam nibh. Suspendisse quis tincidunt nulla. In hac habitasse platea dictumst. Ut sit amet pretium nisl. Vivamus vel mi sem. Aenean sit amet consectetur sem. Suspendisse pretium, purus et gravida consequat, nunc ligula ultricies diam, at aliquet velit libero a dui.</p>

```

Issue 2:
Issue Description

This is for the feature/curriculum-expansion branch only when run locally, and NOT the staging branch.

For several challenges (challengeType: 0, using the code editor with the mobile-phone output layout), the jQuery method .css() doesn't seem to be accessing the styles applied in a style tag in the given code. This is affecting any assert tests that use a pattern of assert($('selector').css('propertyToCheck') == 'valueToCompare', 'message: ...');.Here's an example:

<style>
h2 {
position: relative;
top: 15px;
}
</style>
<body>
<h1>On Being Well-Positioned</h1>
<h2 class="testCode">Move me!</h2>
<p>I still think the h2 is where it normally sits.</p>
</body>
<script>
console.log($('h2').attr('class')); // Works great, returns testCode
console.log($('h2').css('position')); // returns undefined
console.log($('h2').css('top')); // returns undefined
</script>

Related assert tests that aren't passing:

assert($('h2').css('position') == 'relative', 'message: Add a CSS rule to specify that the <code>h2</code> has a <code>position</code> set to <code>relative</code>.');

assert($('h2').css('top') == '15px', 'message: Use a CSS offset to relatively position the <code>h2</code> 15px away from the <code>top</code> of where it normally sits.');

Browser Information

Browser Name, Version: Firefox 48.0.1

Operating System: Mac OS X, El Capitan v10.11.3

Mobile, Desktop, or Tablet:
Issue 1:
pd.date_range() handles closed intervals correctly for daily time series but incorrectly for monthly.

In this example, notice how freq='D' gives 4 days while freq='M' gives 3 months, dropping the final 2017-07-31:

pd.date_range(start='2017-01-04', end='2017-01-07', freq='D', closed=None)
# DatetimeIndex(['2017-01-04', '2017-01-05', '2017-01-06', '2017-01-07'], dtype='datetime64[ns]', freq='D')

pd.date_range(start='2017-04', end='2017-07', freq='M', closed=None)
# DatetimeIndex(['2017-04-30', '2017-05-31', '2017-06-30'], dtype='datetime64[ns]', freq='M')

This is with pandas 0.19.2. Here is my output for pd.show_versions():

pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-71-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.19.2
nose: None
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.11.3
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.2.2
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 2.0.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.5
boto: None
pandas_datareader: None

Issue 2:
#5637 has some examples as well

When the end date in date_range is not a valid business day it appears that date_range first sets it to a valid date before deciding whether to return the last date depending on whether the interval is closed or open on the right. This results in the date range being incorrectly truncated when the interval is open-right - e.g.

In [22]: for d in pd.date_range('20-Mar-2014', '23-Mar-2014', freq='D'):
...: print d.strftime('%a %d-%b-%Y')
...:
Thu 20-Mar-2014
Fri 21-Mar-2014
Sat 22-Mar-2014
Sun 23-Mar-2014

In [23]: pd.date_range('20-Mar-2014', '23-Mar-2014', freq='B')
Out[23]:
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-03-20, 2014-03-21]
Length: 2, Freq: B, Timezone: None

In [24]: # should return all valid business days <= '23-Mar-2014'
...: pd.date_range('20-Mar-2014', '23-Mar-2014', freq='B', closed='left')
<class 'pandas.tseries.index.DatetimeIndex'>
[2014-03-20]
Length: 1, Freq: B, Timezone: None
Issue 1:
Each state hook update causes a seprate re-render in asynchronous code.

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

export default function App() {
const [foo, setFoo] = useState("initial_foo");
const [bar, setBar] = useState("initial_bar");

const onAction = useCallback(async () => {
await sleep(0); // this is the culprit
setFoo("updated_foo");
setBar("updated_bar");
}, []);

console.log(`rendering foo:${foo} bar:${bar}`);

return (
<>
<button onClick={onAction}>run test</button>
<div>{"foo: " + foo}</div>
<div>{"bar: " + bar}</div>
</>
);
}

console logs:

rendering foo:initial_foo bar:initial_bar
rendering foo:updated_foo bar:initial_bar
rendering foo:updated_foo bar:updated_bar

React version: 16.13.1

Steps To Reproduce

Follow the codesandbox link

Press 'run test' button

Check the console

Link to code example:

https://codesandbox.io/s/soextrarerenderswithsleep-3o2g1

The current behavior

Each setState triggers a separate render in asynchronous code, as the console.log indicates.

The expected behavior

All setState should be merged into a single render, the same, as it happens in synchronous code.

Also, this is on Stack Overflow

Issue 2:
Do you want to request a feature or report a bug?

bug + improvement

What is the current behavior?

Sometimes state updates caused by multiple calls to useState setter, are not batched.

sandbox1: https://codesandbox.io/s/8yy0nw2m28

sandbox2: https://codesandbox.io/s/1498n44yr3

Step to reproduce: click on increment all text and check the console

diff:

// sandbox1
const incAll = () => {
console.log("set all counters");
incA();
incB();
};

// sandbox2
const incAll = () => {
setTimeout(() => {
console.log("set all counters");
incA();
incB();
}, 100);
};

console1:

render
set all counters
set counter
set counter
render

console2:

render
set all counters
set counter
render
set counter
render

What is the expected behavior?

Render function should be called once after multiple calls of setState

Which versions of React, and which browser / OS are affected by this issue? Did this work in previous versions of React?

react: 16.7.0-alpha.2
Issue 1:
The following should follow the redirects but it just hangs

import requests
requests.head("http://bit.ly/DyE5", allow_redirects=True)

Have tested this on version 2.6.0, 2.6.1 and 2.6.2 and it stopped working at 2.6.1.

Have also tested as a GET request and that works fine on all versions.

Issue 2:
Starting in 2.6.1 an exception is thrown if a connection with chunked transfer-encoding is reused. I suspect that the bug is in the bundled urllib3 which was upgraded from 1.10.2 to 1.10.3.

#!/usr/bin/env python
import requests
s = requests.Session()
r = s.get('http://httpbin.org/stream/1')
r.json()
r = s.get('http://httpbin.org/stream/1')
r.json()

$ ./test.py
Traceback (most recent call last):
File "./test.py", line 10, in <module>
r = s.get('http://httpbin.org/stream/1')
File "[...]/requests/requests/sessions.py", line 477, in get
return self.request('GET', url, **kwargs)
File "[...]/requests/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "[...]/requests/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "[...]/requests/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ResponseNotReady())
Issue 1:
"For now, as long as I can ask, "get me all the lambda infos into which this function was inlined" (and can go to an fptr from there), that would be most amazing!" #14815 (comment)

(adding this as an issue mostly so I can assign it to myself as a high-priority item for debugging)

Issue 2:
I thought it would be better to submit this as a separate issue in order to bring it to the attention of the wider community. Although a temporary fix for #3420 has been achieved and the Julia compilation on Windows was working fine, I get a new error after the latest update (compiling and linking works fine, the following error appears towards the end of make):

fixme:ntdll:NtFlushInstructionCache 0xffffffffffffffff 0xc80000 524288

abnormal program termination
*** This error is usually fixed by running 'make clean'. If the error persists, try 'make cleanall'. ***
make[1]: *** [/home/theodore/opt/julia-win64/usr/lib/julia/sys.ji] Error 1
make: *** [release] Error 2

This is apparently a Windows specific error, since Julia updates fine on Linux.
Issue 1:
Migrated issue, originally created by anti-social (@anti-social)

After the commit next test case begun to fail:

class FilterUniqueJoinTest(fixtures.MappedTest, AssertsCompiledSQL):
run_setup_mappers = 'once'

@classmethod
def define_tables(cls, metadata):
Table('users', metadata,
Column('id', Integer, primary_key=True,
test_needs_autoincrement=True),
Column('name', String(50)),
Column('date_created', DateTime, default=datetime.datetime.now))

Table('orders', metadata,
Column('id', Integer, primary_key=True,
test_needs_autoincrement=True),
Column('user_id', Integer, ForeignKey('users.id')))

@classmethod
def setup_classes(cls):
users = cls.tables.users
orders = cls.tables.orders

class User(cls.Comparable):
pass

class Order(cls.Comparable):
pass

mapper(User, users, properties={
'orders': relationship(Order, order_by=users.c.id.desc())
})

mapper(Order, orders)

@classmethod
def insert_data(cls):
User = cls.classes.User
Order = cls.classes.Order

sess = create_session()
o1 = Order()
o2 = Order()
o3 = Order()

u1 = User(name='Mike')
u1.orders = [o1, o2]
u2 = User(name='John')
u2.orders = [o3]

sess.add(u1)
sess.add(u2)
sess.flush()
sess.close()

def test_filter_unique_column(self):
User = self.classes.User

sess = create_session()

users = (
sess.query(
User.date_created.label('dt'),
User,
)
.options(joinedload(User.orders))
.all()
)
assert len(users) == 2

def test_filter_unique_func_column(self):
User = self.classes.User

sess = create_session()

users = (
sess.query(
func.date(User.date_created, 'start of month').label('month'),
User,
)
.options(joinedload(User.orders))
.all()
)
assert len(users) == 2

Sqlalchemy 1.0:

test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_column PASSED
test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_func_column PASSED

Sqlalchemy 1.1:

test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_column PASSED
test/orm/test_joins.py::FilterUniqueJoinTest::test_filter_unique_func_column FAILED

Is it expected behaviour?

Issue 2:
Migrated issue, originally created by adridg (@adridg)

With SQL Server, you can create columns of type VARCHAR(MAX). This is the modern rendition of NTEXT or TEXT, and SQL Alchemy translates String() into VARCHAR(MAX). With SQL Server 2012 or later, this happens automatically, and it's described in http://docs.sqlalchemy.org/en/latest/changelog/changelog_10.html#change-f520106ec3455eaa056110c048aa4862 .

However, on reflection (through the inspector) you can end up with a VARCHAR (sql type-)object that has a length of "max" (i.e. a string). Printing that object causes an exception. As a very simple and contrived example:

from sqlalchemy import VARCHAR

print VARCHAR()

print VARCHAR(80)

print VARCHAR("max") # Fails in _render_type_string

Attached find a simple test program that creates a table with a String() column, then reflects it to discover the VARCHAR(MAX) column.

Attachments: t.py
Issue 1:
A bug or perhaps a surprise: in #6053, assignment for structured arrays was changed to be by position rather than field name. However, it seems equality checking was forgotten, leading to the following strange behaviour, where one assigns something but cannot then test equality. It would seem an oversight, so cc @ahaldane, who did #6053.

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.21.0.dev0+215.gd7a75e8e8'

In [3]: a = np.array([(1, 2), (3, 4)], dtype=[('a', 'i'), ('b', 'i')])

In [4]: b = np.zeros(2, dtype='i,i')

In [5]: a
Out[5]: array([(1, 2), (3, 4)], dtype=[('a', '<i4'), ('b', '<i4')])

In [6]: b
Out[6]: array([(0, 0), (0, 0)], dtype=[('f0', '<i4'), ('f1', '<i4')])

In [7]: b[:] = a

In [8]: b
Out[8]: array([(1, 2), (3, 4)], dtype=[('f0', '<i4'), ('f1', '<i4')])

In [9]: a == b
<ipython-input-9-eeadcaeab874>:1: FutureWarning: elementwise == comparison failed and returning scalar instead; this will raise an error or perform elementwise comparison in the future.
a == b
Out[9]: False

In [10]: a == b.astype(a.dtype)
Out[10]: array([ True, True])

p.s. @seberg - is it expected that the cast cannot be done as "safe"? Or is this still in the process of being changed?

In [12]: a == b.astype(a.dtype, casting='safe')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-12-896f15d3d149> in <module>
----> 1 a == b.astype(a.dtype, casting='safe')

TypeError: Cannot cast array data from dtype([('f0', '<i4'), ('f1', '<i4')]) to dtype([('a', '<i4'), ('b', '<i4')]) according to the rule 'safe'

Issue 2:
Describe the issue:

The range of np.int8 is -128 to 127.

The ideal range of np.abs(np.int8) is -127 to 128.

But apparently 128 overflow.

So np.abs(-128, dtype=int8) = -128, while the rest of the cases are all correct.

This error is hard to detect and is also found in np.int16.

Reproduce the code example:

import numpy as np
a = np.array([-128, -127], dtype=np.int8)
print(a)
# [-128 -127]

print(np.abs(a))
# [-128, 127]

Error message:

No response

NumPy/Python version information:

numpy: 1.18.5

sys: 3.7.10 (default, Jun 4 2021, 14:48:32)

[GCC 7.5.0]
Issue 1:
When calling julia on the terminal, the following error is thrown:

PS C:\Users\Martin Cornejo> julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.4.1 (2020-04-14)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

ERROR: Invalid history file (C:\Users\Martin Cornejo\.julia\logs\repl_history.jl) format:
An editor may have converted tabs to spaces at line 46972
Stacktrace:
[1] error(::String, ::Int64) at .\error.jl:42
[2] hist_from_file(::Any, ::Any, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:456
[3] setup_interface(::REPL.LineEditREPL, ::Bool, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:874
[4] setup_interface(::REPL.LineEditREPL; hascolor::Bool, extra_repl_keymap::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:779
[5] setup_interface at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:779 [inlined]
[6] (::Pkg.var"#1#2")(::REPL.LineEditREPL) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:519
[7] __atreplinit(::REPL.LineEditREPL) at .\client.jl:339
[8] #invokelatest#1 at .\essentials.jl:712 [inlined]
[9] invokelatest at .\essentials.jl:711 [inlined]
[10] _atreplinit at .\client.jl:346 [inlined]
[11] (::Base.var"#764#766"{Bool,Bool,Bool,Bool})(::Module) at .\client.jl:382
[12] #invokelatest#1 at .\essentials.jl:712 [inlined]
[13] invokelatest at .\essentials.jl:711 [inlined]
[14] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at .\client.jl:367
[15] exec_options(::Base.JLOptions) at .\client.jl:305
[16] _start() at .\client.jl:484

[ Info: Disabling history file for this session
julia>

julia> versioninfo()
Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-3630QM CPU @ 2.40GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-8.0.1 (ORCJIT, ivybridge)

Issue 2:
It would be useful if the REPL allowed configuration of the default IO context, so that users can customize how things are displayed in their .juliarc.
Issue 1:
Justin Mills opened SPR-6752 and commented

Using RestTemplate to connect to a server that occasionally throws HTTP Status of 420, I am getting IllegalArgumentExceptions when accessing the HttpStatus object in my implementation of the ResponseErrorHandler interface. When I removed that, and started pulling the status from the Headers directly, RestTemplate started throwing errors due to the logger.warn call in RestTemplate.handleResponseError(). I've gotten around this by not using the HttpStatus enum (getting status from the HttpHeaders directly) and by turning up the log level for RestTemplate to ERROR. Shouldn't this support any HTTP status value?

Affects: 3.0 GA

Issue Links:

#13707 Implementation of HttpStatus as an enumeration precludes other status codes ("is duplicated by")

#13949 In order to handle non standard HTTP Status codes returned by services in RestTemplate, I would like to add support for handling non standard status codes and passing on responses for these responses to callers ("is duplicated by")

#14042 Unknown status codes (i.e. not in HttpStatus enum) prevent HttpClientErrorException and HttpServerErrorExceptions from being raised

#20529 RestTemplate doesn't consistently tolerate unknown HTTP status codes

Referenced from: commits 68b4c7d

Backported to: 3.0.1

Issue 2:
Kenny MacLeod opened SPR-4235 and commented

When I use the new @Controller and @RequestMapping annotations, and the target method throws an Exception, it isn't reported very well by the framework.

Take this example stack trace fragment, caused by an IOException theown by my controller:

java.lang.IllegalStateException: Unexpected exception thrown by method - java.io.FileNotFoundException: \tmp\2007\10\csv\top_100_stops_wap.csv (The system cannot find the path specified)

at org.springframework.util.ReflectionUtils.handleInvocationTargetException(ReflectionUtils.java:210)

at org.springframework.util.ReflectionUtils.handleReflectionException(ReflectionUtils.java:190)

at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:168)

at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:245)

at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)

at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)

You get the message text of the original exception, but the stack trace isn't much use for diagnosing the problem.

Can ReflectionUtils be modified to propagate or log the original exception?

Affects: 2.5 final

Issue Links:

#9014 {portlet-mvc, annotation based} SimpleMappingingExceptionResolver does not handle caught exceptions correctly ("is duplicated by")
Issue 1:
I have searched the issues of this repository and believe that this is not a duplicate.

I have checked the FAQ of this repository and believe that this is not a duplicate.

Environment

Dubbo version: latest

Operating System version: linux

Java version: 9+

Requirement

After Java 9 ，the default JVM garbage collector is G1 , So please add G1 configuration in

dubbo/dubbo-container/dubbo-container-api/src/main/resources/META-INF/assembly/bin/start.sh

when your jdk is higher then Java 8

Issue 2:
I have searched the issues of this repository and believe that this is not a duplicate.

I have checked the FAQ of this repository and believe that this is not a duplicate.

Steps to reproduce this issue

Run AbstractMonitorFactoryTest.testMonitorFactoryGroupCache() or DubboMonitorTest.testCount() many times(may be less than 10 is enough), you will get junit test failed.

Cause Analysis

AbstractMonitorFactoryTest.testMonitorFactoryGroupCache()

In AbstractMonitorFactory.getMonitor(), if we call thenRunAsync() first, then call FUTURES.put(),

in MonitorListener.run, FUTRUE.get() may get null, then the junit test failed.

DubboMonitorTest.testCount()

In DubboMonitor constructor, we starter a schedule that call send() every 20 milliseconds!!

In testCount(), we collect() first, then send(), and send() every 20 milliseconds backend, but we will reset data in send() everytime, so let's consider this sequence: we call collect(), and then call send(), and before Assertions.assertEquals... , call send() backend by the schedule to reset the statisticsMap.

I describe it in Chinese again:

这个问题是偶现的, 但是可以通过多次执行testCount()来复现(执行最多10次一般就能复现出来), 原因是, 在构造Du bboMonitor 的时候, 会启动一个定时器, 每个20毫秒来执行send()方法, 且在send()设置值以后, 就会reset 全局的statisticsMap, testCount()中考虑这么一个执行顺序: 先执行collect(), 在执行send(), 然后在执行诸多的Assertions.assertEquals之前, 可能会执行每20毫秒执行一次的send(), 在这个send()方法执行完成后, statisticsMap就会被重置, 导致junit test失败
Issue 1:
Uncaught Error: spawn /usr/share/atom/atom (deleted) ENOENT

Atom Version: 0.161.0

System: linux 3.13.0-24-generic

Thrown From: Atom Core

Steps To Reproduce

...

...

Stack Trace

At events.js:85

Error: spawn /usr/share/atom/atom (deleted) ENOENT
at exports._errnoException (util.js:742:11)
at Process.ChildProcess._handle.onexit (child_process.js:1051:32)
at child_process.js:1142:20
at process._tickCallback (node.js:378:11)

Issue 2:
TEMPORARY WORKAROUND: ln -s /usr/share/atom/atom "/usr/share/atom/atom (deleted)"

Uncaught Error: spawn /opt/atom/atom (deleted) ENOENT

Atom Version: 0.152.0

System: linux 3.13.0-40-generic

Thrown From: Atom Core

Steps To Reproduce

...

...

Stack Trace

At events.js:85

Error: spawn /opt/atom/atom (deleted) ENOENT
at exports._errnoException (util.js:742:11)
at Process.ChildProcess._handle.onexit (child_process.js:1051:32)
at child_process.js:1142:20
at process._tickCallback (node.js:378:11)
Issue 1:
🚀 Feature

Allow dataloader to be chunkable. Meaning, we can break each epoch into N chunks.

Motivation

for users who have large training set, the pattern often looks like this

def train_model(self)
while True:
self.train()
self.iterations += 1

if self.iterations % VAL_ITER == 0:
self.val()

This allows the user the evaluate many times per epoch. However, this means that there will be 2 active dataloader at a time.

A more ideal set up would be, to allow the user to specify the length of each chunk of dataloader. So the user can specify that, he/she wants to evaluate every N number of examples. So the train dataloader will exit before the val dataloader will begin. And when validation finishes, it will initialize a new train dataloader to train the next chunk

Pitch

Currenlty, it means that there will be 2 active dataloader at a time. Which doubles the RAM usage.

Supporting this feature will half the memory usage for many users

Issue 2:
❓ Questions and Help

If want to add Bert for pytorch community, can I put the models and weights of BERT into the repository of pytorch/text ?

As pytorch/vision is for Models specific to Computer Vision, but i can't find any NLP models in pytorch/text