Silviase/jgov_v1.4_full_ann huggingface dataset:
where EDi is a vector dummy variable indicating educational attainment and $ \gamma $ is a vector of associated coefficients. This provides a summary across educational categories of the percentage change in the mean tenure relative to the 1944 birth cohort $ (100 \times [\exp(\widehat{\alpha_{j}} - \widehat{\alpha_{1944}}) - 1]) $ controlling educational distribution over time. The estimation result based on (2) is reported in Figure 4, but the analysis result is limited to the ESS because the BWSs does not record part-time workers' educational background. After adjusting the impact from the changes in educational attainment, the general decreasing trend seems to show almost the same pattern, by comparing Figure 4 and Figure 3. The decreasing rate stays at almost the same level (i.e. about a 32% between the 1944 birth cohort and the 1981 birth cohort for male workers). Although the general trend is almost similar, the decreasing rate is even greater for female workers, with the control of changes in educational attainment. This implies that changes in the distribution of educational attainment have actually worked in the opposite direction of what was expected. In other words, while longer educational attainment would directly imply shorter mean tenure, in the case of women it contributed to make mean tenure longer among recent birth cohorts. This might mean that women's educational advancement led to the creation of positions to make them more stable, or possibly to make them more committed to the labor force.

In addition to changes in educational attainment, there may also be the fact that recent birth cohorts are more likely to spend several years looking for a job that fits well, instead of finding a permanent job upon graduation. This leads to a shift in the whole tenure distribution to the left-hand side, and to a decrease in mean tenure levels for every birth cohort.

### 3.3 Proportion of Permanent-Regular / Nonpermanent-Regular Workers

The second and more important factor that could account for the decline in mean tenure would be the increase in the share of non-regular employees. We define permanent-regular workers as those who are employed, working either as a regular employee or an executive official, under a permanent contract that includes an indefinite contract or a fixed-term contract that extends one year or more. Nonpermanent-regular workers include all other types of workers. They correspond to those who are employed, working either as a temporary/daily worker, or a permanent worker who is categorized as a part-time worker (including both Paat and Arubaito), temporary worker, or contract worker. We divide the sample into permanent-regular workers and nonpermanent-regular workers, and re-estimate (1) by sex.
Before this estimation, to examine if the proportion of permanent-regular workers has really declined and to what extent over birth cohorts, we estimated the probability of being a permanent-regular worker on the birth-cohort indicators by sex, with the control for ages. The model is

$$ \mathrm{PD}_{\mathrm{ijk}}=\mathrm{C}_{\mathrm{j}}\alpha+\mathrm{A}_{\mathrm{k}}\beta+\varepsilon_{\mathrm{ijk}}\ (3) $$

where PD is an indicator for a permanent-regular worker. From model (3), we can estimate the birth-cohort effects on the probability of being a permanent-regular worker with the control for ages (we call these effects the relative fraction of permanent regular workers). Figure 5 Panel A shows the estimated relative fraction for men, using both the ESS and the BWS, and Figure 5 Panel B corresponds to those for women. The shapes are somewhat different by sex, but both are declining among recent birth cohorts.

By comparing Figure 5 Panels A and B, we can see that the declining speed among male workers is much slower than that among female workers, and the speed starts to accelerate from the recent birth cohorts (i.e., 1970 birth cohorts and later). This might be related to the fact that female workers tended to participate in the workforce more as nonpermanent-regular workers among the more recent birth cohorts. The drastic decline for both male and female workers after the 1970 birth cohorts could be related to the bad macroeconomic conditions when they left the university and looked for jobs. Female workers could have been affected to a greater extent, facing more difficulties in finding a job as a permanent worker than male workers. In Figure 5 Panels A and B, BSWS and ESS result show similar trends among cohorts, while the relative fraction of recent male cohorts are estimated to be lower with ESS results than BSWS ones.

As we have observed that there are rapid increases in the relative fraction of nonpermanent-regular workers in recent birth cohorts, we consider that this trend might have affected the average tenure by birth cohort, as such workers are likely to have higher turnovers than permanent-regular workers. To see if this holds, we re-estimate relative tenure of each birth cohort by dividing the data into permanent-regular workers and nonpermanent-regular workers.

The re-estimation result of (1) is included in Figure 6 Panels A (male) and B (female) for permanent-regular workers and in Figure 7 Panels A (male) and B (female) for nonpermanent-regular workers. For male workers, the trend of relative tenure by birth cohort is similar for “all workers” (Figure 3 Panel A) and “permanent-regular workers,” as a large proportion of workers are permanent-regular workers. On the other side, the trend of relative tenure mostly coincides with that of nonpermanent-regular workers for
female workers (Figure 3 Panel B). The results indicate that the relative average tenure is declining among recent cohorts, even among permanent-regular workers. Therefore, although we assumed that the increase in the proportion of nonpermanent-regular workers among recent cohorts would explain the shorter average tenure, we found that this is not sufficient to explain the decreasing trend of the average tenure among male workers.

We should note that the estimation results look different between the ESS and the BSWS in all cases. For male permanent-regular workers, the relative tenure is estimated to be lower for the more recent cohort, but the pace of decline is more distinct for the ESS than the BWS. The trends of non-permanent male workers are very different between the ESS and BWS results, as the declining trend among recent cohorts is obvious in the ESS but not in the BWS results. The trends of permanent female workers look similar, but BWS estimates show a greater increase and decrease in average tenure by birth cohorts. The results of nonpermanent female workers are the opposite: BWS results indicate that the tenure became relatively longer among more recent birth-year cohorts, while ESS results show that it became shorter among them.

As discussed in an earlier section, part of these differences between the ESS and the BSWS may have been caused by the difference in data coverage. Any consistent relationships between i) birth cohorts, ii) data coverage, and iii) tenure length might explain the discrepancy between ESS and BSWS estimation results to some extent. For example, if (i) the share of workers employed at new establishments, (ii) that of workers employed at very small establishments, or (iii) that of workers in non-manufacturing sectors has been increasing among more recent cohorts, then they are worker groups that are not/less likely to be covered in the BSWS compared with the ESS. Assuming that such workers tend to have short tenure, the sample coverage would be able to explain the observed discrepancies.

By comparing male and female results, we should note the difference observed in permanent workers. The general trend among cohorts seems to be quite opposite: The results for female permanent workers show an increasing trend except for the most recent cohorts, while those for male permanent workers have a decreasing trend. The increasing trend may imply the stronger commitment to the labor force among women as permanent-regular workers after a high-growth period of Japan's economy. At the same time, we can also think of demand-side factors that contribute to the recent decline of both sexes. For example, firms began to hire more permanent-regular workers with short-contract terms after the mid 1990s, and many young workers, both male and female but more obviously the latter, were absorbed into such posts. We assume that the
recent declining trend for nonpermanent-regular workers derived from ESS results may be related to the fact that individuals formed such contracts as contract workers or dispatched workers, whose tenure is even more unstable compared to the rest of nonpermanent-regular workers (e.g., contingent workers).

The difference between ESS and BWS estimation results for female workers might be interpreted as the difference in employment behaviors between medium/large establishments and small ones. With regards to permanent regular female workers, the relative tenure has increased for the 1950-1970 birth cohorts, particularly at establishments of a certain size and beyond. The trend of nonpermanent-regular female workers is the opposite, which might indicate that the status of such workers has become more unstable than before, particularly for workers employed at very small establishments or in certain industry sectors.

In the next section, we examine to what extent the difference in the coverage of the two surveys can explain the discrepancy in the estimation results of relative tenure.

### 4. Filling the Gap between the ESS and the BSWS

### 4.1 Differences in Estimation Results and Data Coverage

As discussed in the previous section, by comparing ESS and BSWS results, differences in relative tenure are particularly obvious among recent cohorts. We considered two possible explanations for such findings: i) an increase of workers with particularly short tenure, after controlling age, among recent cohorts, which is particular in the ESS and not in the BSWS, and ii) an increasing discrepancy in the sample coverage between the ESS and the BSWS among recent cohorts regarding workers with short tenures.

## i) Shift of the tenure distribution among recent cohorts

The discrepancy of ESS and BSWS results could be caused by the BSWS failing to capture workers with particularly short years of tenure because of their contractual form. If this is the case, the distribution of tenure estimated from the ESS should have a fatter left-tail than that from the BSWS. We examined whether such difference of the distributions is actually observed by comparing the distributions of the residuals derived from estimation (1) from the ESS and the BSWS, among two different birth cohorts. Figure 8 (male) and Figure 9 (female) describe the residual distribution by sex, derived from ESS and BSWS estimations, for birth cohorts of 1945-1950 and 1965-1970. For both the ESS and the BSWS, residual distributions for male workers are skewed to the left, implying that many workers share the same length of tenure that is slightly above the average tenure because the majority of male workers work for the same company.
after graduating from school. This skewed distribution is less prevalent for the female residual distribution.

The shapes of the ESS and BSWS residual distributions of the birth cohorts of 1945-1950 look quite similar for both male and female cases, while the distributions of the 1965-1970 cohorts are thicker around the left-hand side (i.e., left-skewed) for males and females in the ESS than in the BSWS. The results provide us an intuitive explanation of why the relative tenures are estimated as lower among recent cohorts in the ESS estimation than in the BSWS one.

## ii) Increase in the sample discrepancy among recent cohorts

We assume that such differences may have been caused by the missing samples that tend to have short tenures from the BSWS, which are covered by the ESS. As discussed in i) above, recent cohorts in ESS samples are more likely to have shorter tenure than BSWS ones, after controlling for age effects. Based on earlier findings from Table 3, we focus on differences in industry and firm-size distributions between the two datasets. We therefore create a weight for a sample to be included in the BSWS from the combined dataset of the ESS and the BSWS. Specifically, we construct the following weight to recover the industry and firm-size distribution in the BSWS using the ESS:

$$ \mathrm{w}_{\mathrm{i}}=\frac{\mathrm{P}(\mathrm{B S W S}=1|\mathrm{x}_{\mathrm{i}})}{1-\mathrm{P}(\mathrm{B S W S}=1|\mathrm{x}_{\mathrm{i}})}\cdot\frac{1-\mathrm{P}(\mathrm{B S W S}=1)}{\mathrm{P}(\mathrm{B S W S}=1)}, $$

where BSWS is an indicator variable that takes one if the observation is from the BSWS and zero if the observation is from the ESS. The vector $ x_{i} $ includes 88 group dummy variables constructed from 11 industry categories and 8 firm-size categories listed in Table 3. The probability to be included in the BSWS, which is $ P(BSWS = 1|x_{i}) $ , is estimated by a linear probability model applied to the pooled data of the ESS and the BSWS for each year. Since BSWS counterparts are not available for the 1982 and 1987 ESS, the 1989 BSWS is used for those two years. The estimation of the linear probability model does not violate the notion of probability because the model is a saturated model.

We expect that the weight distribution shifts to the left for the ESS samples of the recent survey years, implying that more observations appear only in the ESS but not in the BSWS. Figure 10 shows the distributions of estimated weights by sex for years 1982 and 2007. We observe a significant fraction of observations with zero weight that correspond to the observations that appear in the ESS but not in the BSWS. From the
figure, it is not quite obvious whether our assumption that the weight distribution would be more skewed to the left for 2002 samples than 1987 ones (i.e., more samples that are not/less likely to be covered by BSWS exist in the 2002 survey than in the 1987 survey) holds.

In summary, we consider that the above examination results at least do not deny the possibility that the sampling discrepancy explains the differences in the estimation results of relative tenure between the ESS and the BWS. We then examine relative tenures, adjusting the differences in sampling frequency for the ESS case.

### 4.2 Estimation Results Based on the ESS with Weights

Figures 11 and 12 show the results of the estimation (3) by using i) the ESS, ii) the BSWS, and iii) the ESS with weights by sex for permanent-regular and nonpermanent-regular workers, respectively. Regarding the results for male permanent-regular workers, the results from the third estimation almost trace the first estimation results, with a slight adjustment in the direction closer to the second one (Figure 10). Thus even after adjusting sample weights, the estimation results still exhibit almost the same level of discrepancy as before. This indicates that the difference between the ESS and the BSWS results have not been caused by the difference in the industry and firm-size structure between samples of the two surveys, as this factor can explain the difference to a very limited extent. This could have been caused by the BSWS's survey structure, which might tend to capture only a certain group of workers within establishments, causing the higher estimation results for the average tenure length than those of the ESS.

For the female permanent-regular workers, the estimated trend is volatile, and the relationships among the three different estimation results are not as clear as in the male case (Figure 11). In this case, we can at least assume the sample coverage difference explains the discrepancy between ESS and BSWS results for workers from the 1960-1970 birth cohorts.

Figure 12 shows the results for nonpermanent-regular workers. The adjustment of ESS samples by weights does not reduce the discrepancy between the ESS and the BSWS in a substantive way, and it simply traces the ESS results without weights. This result implies that in the case of nonpermanent-regular workers, the difference in industry or firm-size distribution among samples of the two surveys has not been a cause of the different estimation results: Those who are surveyed in the BSWS tend to have an increasing (female) or stable (male) trend of average tenure by birth cohorts, while those covered in the ESS tend to have a decreasing trend.
To summarize, we find that there are still substantial differences in the average tenure between the adjusted ESS and BWSs among male permanent-regular workers, as well as among nonpermanent-regular workers of all cohorts. Assuming that the ESS sample indeed represents the whole population of Japan, the above results indicate certain factors that contribute to the BWSs's failure to capture the recent cohorts of permanent regular workers with short tenures through its sampling process. One such factor may be the sampling process of workers at each establishment by using payroll records. $ ^{9} $ The workers of the BWSs are designed to be randomly selected among all workers enrolled on the payroll books, while the worker-selection process at an establishment level may not be perfectly random for certain reasons, such as: i) the payroll books are supposed to be created each time employees are paid-while most workers are paid regularly (e.g., monthly) at the same time, there may be a group of workers under contracts that make them get paid on an irregular basis-such workers might not be covered in the sampling process, ii) there might be a multiple number of payroll books per establishment, and some of the books may be kept and updated by a section that is not the same as the one that is responsible for responding to the BWSs. The fact that the BWSs asks the annual bonus amount in the last year may also create a bias to sample workers with at least one year of tenure.

After all, the purpose of the BSWS is to estimate wage conditional on workers' characteristics such as age, education, and tenure, and the sampling is designed to serve this purpose efficiently. Thus, the fact that those conditioning variables of the BSWS do not represent the population is not surprising.

These are just conjectures that require further evidences. We can at least conclude that we should not solely depend on the BSWS to discuss recent trends in employees' average tenure in Japan, as the survey may not represent the whole workforce, particularly in that it misses certain types of workers with short tenures. Among others, we would like to highlight our findings from the ESS that the average tenure is decreasing among recent cohorts for both permanent-regular and nonpermanent-regular workers, for both male and female workers.

### 5. Falling tide sinks all boats

The results based on the ESS heretofore clearly indicate the decline of long-term
employment among younger-born cohorts in Japan. The model specification heretofore assumed that a worker who belongs to a specific cohort experiences a uniform decline of employment stability across ages over his/her career. It is possible, however, that the declining stability is concentrated only at the early career stage and, after an initial job-hopping period, workers may enjoy long-term employment as before. Alternatively, only middle-aged workers may have experienced a declining long-term employment relationship because of layoffs of mid-career workers. These possibilities are assessed by estimating the same regression model using subsamples divided by age. While our ability to implement this identification strategy is limited by the data, as it requires several cohorts observed at the same age, we roughly divide the sample into workers whose ages are between 25 and 39 and between 40 and 54. The estimation results are summarized in Figure 13. For male workers, the percentage decline of the average tenure of workers among 25-39 year olds and 40-54 year olds are very similar, suggesting that typical workers experience less job stability over their career regardless of age. The estimation results for female, in contrast, indicate that the extension of average job tenure between 1945 and 1960 birth cohorts was observed only at the ages between 25 and 39. Those aged between 40 and 54 have experienced a monotonic decline of average job tenure across birth cohorts.

One might argue that the declining trend is observed because the industry composition of workers has changed over the years. As evidenced in Table 3, the industry composition has changed over the last 20 years, and this implies that younger cohorts are more likely to be in certain industries than previous cohorts. Because the means of workers' tenure differ across industries, a change of industry composition across cohorts mechanically alters the mean tenure of workers across birth cohorts. To access this possibility, Figure 14 graphs changes in age-specific mean tenure by six sectors and by sex. Even though there are differences in the level of changes by sector (for example, the information and transportation sector records the greatest level of negative changes for all birth cohorts, while the manufacturing and construction sector shows relatively smaller changes for all cohorts), we note that all sectors experience a declining trend of changes in age-specific mean tenure across birth cohorts, for both male and female workers.

One may also point out possible heterogeneous trends of long-term employment across firm sizes. Previous studies found that long-term employment is more prevalent in larger firms (Raisian and Hashimoto (1985) and Ariga, Brunello and Ohkusa (2000)). If the value of keeping long-term employment is high for larger firms, workers in larger firms may have been relatively immune from declining trend of long-term employment.