Silviase/jgov_v1.4_full_ann huggingface dataset:
data of 68 industries for every five-year period from 1980 to 1998 and estimated a fixed effect model. They found that the increase in the deregulation index has a significant positive effect on that industry’s TFP growth.

We have seen that in the 1990s substantial deregulations were accomplished in the non-manufacturing industries, especially in communication, wholesale and retail trade, and finance, insurance and real estate, and this change seems to have contributed to the acceleration of TFP growth in these industries. But we should also note that compared with other developed countries Japan’s TFP growth in the non-manufacturing sector is still quite low.

Table 4.3 compares the TFP growth rates during the 1990s in major service sectors in Japan, Australia and the U.S. $ ^{20} $ TFP growth rates in Australia are taken from McLachlan, Clark, and Monday (2002). TFP growth rates in the US are taken from Yoshikawa and Matsumoto (2001). In these studies value added is used as a measure of output. For this international comparison we calculated Japan’s value added based TFP growth rate of the service industries from the JIP Database. Compared with Australia, Japan’s TFP growth rate in the 1990s was lower in six out of nine industries. Compared with the US, Japan’s TFP growth rate in the 1990s was lower in five out of eight industries. And in the case of average of service industries, Japan’s TFP growth is still lower than that of the other countries.

The most developed economies have experienced a shift in the production structure from manufacturing to services. Hence, in order to maintain the pace of TFP growth in the economy as a whole, an acceleration of TFP growth in the service industries is very important.

#### 4.2. Decomposition Analysis of TFP Growth in the Manufacturing Sector
Next let us consider the slowdown of TFP growth in the manufacturing sector.

As Baily, Hulten and Campbell (1992) and Foster, Haltiwagner and Krizan (1998) have shown in their productivity decomposition analysis, the start-up of productive establishments and the closure of unproductive establishments substantially contributed to US TFP growth. As Figure 4.1 shows, the start-up rate (number of newly opened establishments/number of all establishments) and the closure rate in Japan are about one half of the corresponding values for the US in the 1980s. Moreover, the gap has widened in 1990s. In particular, the start-up rate in Japan’s manufacturing sector has declined to about 2% in recent years. Probably this factor has contributed to the slowdown in TFP growth in Japan’s manufacturing sector.

Using firm level data of the Ministry of International Trade and Industry’s Kigali Katsudo Kihon Chosa (Basic Survey on Business Activities by Enterprises), Fukao and Kwon (2003) studied this issue. $ ^{21} $ Adopting the methodology used by Baily, Hulten and Campbell (1992) and Forster, Haltiwanger and Krizan (1998), they decomposed the manufacturing sector’s TFP growth of the 1994-98 period into the following five factors. $ ^{22} $

Within effect: $ \sum_{i\in s}\theta_{it-\tau}\Delta\ln TFP_{it} $ ,

$$ \mathrm{B e t w e e n~e f f e c t:~}\sum_{i\in s}\Delta\theta_{i t}(\ln T F P_{i t-\tau}-\overline{\ln T F P_{t-\tau}}), $$

Covariance effect: $\sum_{i\in s}\Delta\theta_{it}\Delta\ln TFP_{it}$,

Entry effect: $\sum_{i\in N}\theta_{it}(\ln TFP_{it}-\overline{\ln TFP_{t-\tau}})$ and
$$ \mathrm{E x i t~e f f e c t:}\sum_{i\in X}\theta_{i t-\tau}\left(\overline{\ln T F P_{t-\tau}}-\ln T F P_{i t-\tau}\right), $$

where $ \rho_{it} $ denotes firm i's sales share in year t. $ TFP_{it} $ denotes firm i's TFP level in year $ t^{23,24} $ . $ TFP_{t} $ with an upper bar denotes the industry average TFP level. N is the set of newly entered firms and X, the set of exited firms.

Fukao and Kwon's (2003) decomposition result is reported in Tables 4.4 and 4.5. $ ^{25} $ Following preceding studies, they conducted a decomposition for the upturn period (1994-96) and for the downturn period (1996-98) separately. Table 4.6 compares their results with those of preceding studies for the US and South Korea.

Their major findings are as follows.

1. The exit effect of the whole manufacturing sector in 1996-98 was negative and substantially contributed to the decline in TFP growth in the manufacturing sector. The negative exit effect means that the average TFP level of exiting firms is higher than that of staying firms.

2. The entry effect was positive both in the upturn and the downturn period. But as a result of the low entry rate, the size of the entry effect was not large.

3. The redistribution effect - that is, the share effect plus the covariance effect - was positive but relatively small in comparison with the US.

4. the within effect, i.e. the effect of TFP growth within staying firms, was the largest factor among
all the effects. And this effect changed pro-cyclically.

The above results seem to indicate that the promotion of new entries and making both the exit process and the reallocation process of resources more efficient are very important for an acceleration of TFP growth in Japan’s manufacturing sector. These factors, moreover, are closely related with the allocation of funds through the financial system. Therefore, the problems in Japan’s banking system are likely to have contributed to the slowdown of Japan’s TFP growth and their solution forms an integral part of any attempt to raising the TFP growth rate. $ ^{26} $

### 5. Conclusions

Using the newly compiled data, the Japan Industrial Productivity (JIP) Database, we analyzed Japan’s sectoral TFP growth in recent years. Let us summarize our main results.

1. After taking account of the quality of labor and the capacity utilization rate we found the decline in TFP growth at the macro-level from the 1980s to the 1990s not to be so great. The decline in TFP growth from the 1983-91 period to the 1991-98 period is 0.20 percentage-points. Hayashi and Prescott (2002) seem to have overestimated the size of the TFP growth decline.

2. On the other hand, there was a substantial change in the pattern of sectoral TFP growth. The slowdown in TFP growth mainly occurred in the manufacturing sector. The manufacturing sector’s contribution to macro-TFP growth declined from 0.74 percentage-points in 1983-91 to -0.03 percentage-points in 1991-98. In contrast, TFP growth in the non-manufacturing sectors accelerated during the 1990s. Non-manufacturing sectors’ contribution to macro-TFP growth increased from -0.34 percentage-points in 1983-91 to 0.22 percentage-points in 1991-98.
3. In the 1990s, substantial deregulations were accomplished in non-manufacturing industries, especially in communication, wholesale and retail trade, and finance, insurance and real estate, and this change seems to have contributed to the acceleration of TFP growth in these industries. But we should also note that, compared with other developed countries, Japan’s TFP growth in the non-manufacturing sector is still quite low.

4. Regarding the manufacturing sector in the 1990s, the following three factors seem to have contributed to the low level of TFP growth. First, new entries were very limited. Secondly, the exit effect was negative, that is, the average TFP level of exiting firms was higher than that of staying firms. Thirdly, the reallocation effect of resources was small.
## Appendix A: Data Sources and Estimation Methods of the JIP Database

In this appendix we briefly explain how the JIP Database is compiled.

#### A.1. Estimation of Real Net Capital Stock by Industry and by Capital Goods

To construct real net capital stock by industry and by capital goods, we begin by estimating the net capital stock in 1970 as a benchmark. For the capital stock from 1971 to 1998, we used the perpetual inventory method, making use of the series for annual capital formation by industry and by capital goods and applying a constant depreciation rate for each type of fixed capital stock.

All real series are valued at 1990 prices. Our database consists of 84 industries based on SNA-IO published by the Economic and Social Research Institute (ESRI). As for capital goods, we arrange 37 capital goods in our database based on the commodity flow data in ESRI of the Japanese government. We name our own industry and capital goods classification as in the JIP classification. Our capital stock database covers not only the private sector but also the public enterprise sector and the government service sector. In addition, it includes residential stocks.

## Estimation of Benchmark Capital Stock Data (for 1970)

We construct the benchmark stock by industry and by capital goods based on the National Wealth Survey of 1970. We transform the original data in the following four processes.

First, the statistics in the National Wealth Survey of 1970 are compiled in terms of firms and organizations. On the other hand, the sectoral statistics in the Fixed Capital Formation Matrix, which we used as the most basic statistics for our estimation of capital formation series, are compiled in terms of production activities. In order to make adjustments for this difference in the two statistics, we transformed the original data of the National Wealth Survey of 1970 into activity-based data by making use of the information on the distribution of each asset among sectors, which is available in the Fixed Capital Formation Matrix of 1970.
Second, the sectoral classification in the National Wealth Survey of 1970 is rougher than the JIP industry classification. Therefore, we construct the benchmark stock data which corresponds to the JIP industry classification by using the production data in the Input –Output Table for 1970 or the employee data in the Establishment Census of 1969 and 1972.

Third, the original data in the National Wealth Survey of 1970 are nominal values. Using price deflators for capital goods in the commodity flow statistics in ESRI, we converted the nominal values into values at 1990 price.

Fourth, in the National Wealth Survey of 1970, the statistics on public sectors are for the end of the fiscal year of 1970. Using data on investment flows, we converted the statistics to a calendar year basis.

## Estimation of the Capital Formation Series

We estimate the capital formation series from 1970 to 1998 by industry and by capital goods. Classifications of industry and capital goods are based on the JIP classifications. We construct the capital formation series by the following three steps: (1) We estimate the capital formation series by industry, (2) the capital formation series by capital goods, and (3) the fixed capital formation matrix every year based on capital formation data constructed in (1) and (2). In the following subsections, we will explain each estimation method in detail.

## Estimation of Capital Formation Series by Industry

In the manufacturing sector, we compile the annual series of the capital formation using the Census of Manufacturing. In the non-manufacturing sector, we construct the data by examining statistics in each industry or closing accounts of public enterprises. These statistics are based on sample surveys and do not cover all establishments in each industry.

Next, using data of the Fixed Capital Formation Matrix, which is more reliable but only available every five years, we adjusted the above annual series of capital formation.
## Estimation of the Capital Formation Series by Capital Goods

Basically, we compiled the capital formation series by making use of the commodity flow data of ESRI. The commodity flow data is arranged in an eight-digit classification system. We rearrange this data into the JIP capital goods classification.

The commodity flow data do not include data on construction and buildings which are classified in the JIP capital goods classification Nos. 32-37. We estimate the capital formation series for these capital goods using mainly the statistics published by the Ministry of Land Infrastructure and Transport. Finally, using the Fixed Capital Formation Matrix, we adjusted the above capital formation series by industry.

We should note that our database does not cover capital formation of intangible assets, because it is based on 68SNA.

## Estimation of the Annual Series of Fixed Capital Formation Matrix

As we have explained above, we obtained annual capital formation data by industry or by capital goods. However, we do not have a fixed capital formation matrix for non-benchmark years. We estimated the fixed capital formation matrix for the intermediate years by the RAS method.

## Construction of Real Net Capital Stock for 1970-1998

The fixed capital formation estimated in section 2 is expressed in nominal terms. We convert the series in nominal terms into 1990 prices by using deflators in the commodity flow data of ESRI.

Next, we accumulate capital stock from the benchmark stock in 1970 by the perpetual inventory method. Using this method, we have to consider depreciation. We assume a constant depreciation rate for each capital good. We use the depreciation rate adopted by the Bureau of Economic Analysis in the U.S.

## Estimation of IT Capital Stock
IT capital goods consist of two types: Tangible assets (hardware) and intangible assets (software). Our definition of IT capital goods is similar to that used by the Bureau of Economic Analysis of the US. government. Tangible IT assets include office machines, computers, computer peripherals, communications equipment, optical instruments and medical instruments. $ ^{27} $

In the National Accounts of Japan, only order-made software investment is estimated by making use of the Survey on Specified Service Industries. In countries like the US, the UK and Australia, GDP statistics cover in-house software and general application software as well as order-made software. Making use of the Survey on Information Processing and the Survey on Specified Service Industries, we estimated software investment in Japan in a fashion which is comparable to that of the US, the US and Australia.

Aggregate IT investment including software investment in Japan increased by 12.4% per annum from 1970 to 1998 (Figure 4-1), exceeding the average growth rate of total investment (3.2%). The ratio of IT investment to total investment increased from 28% in 1970 to 31.4% in 1998.

However, it did not increase uniformly like US IT investment. In the early 1990s, its growth stagnated. Probably the stagnation was caused by the following two factors. First, investment in tangible IT assets except computers and computer peripherals was strongly affected by business cycles. Second, investment in in-house software did not increase in the early 1990s, because Japanese firms have reduced costly in-house software and made an effort to increase outsourcing or
utilize more standardized software since the bubble collapsed.

The IT capital stock also increased rapidly. In 1970, the IT capital stock at 1990 prices was only 5.6 trillion yen. In 1998, it reached 136 trillion yen. It grew at 11.4% per annum over this 28-year period. The real growth rate was similar to the nominal growth rate until 1990. However, the price fall in tangible IT capital goods contributed to the real growth of IT capital stock in the 1990s.

#### A.2. Estimation of Labor Input by Industry and by Type of Labor

## Data Description

Our measures of labor input in the JIP Database are constructed by combining the value estimates from the input-output table matrices and data from several labor force surveys. We constructed a detailed data set of the number of workers $ N_{ljt} $ , hours worked $ H_{ljt} $ , and the hourly wage $ W_{ljt} $ (l: type of worker, j: sector, t: year).

We divide the work force cross-classified by sex, age and educational attainment.

Sex (2): Male, Female.

Age (15): 15-19, 20-24, 25-29, 30-34, 35-39, 40-44, 45-49, 50-54, 55-59, 60-64, 65-29, 70-74, 75-79, 80-84, 85-.

Education (4): Junior high school, High school, College, University or more.

Status (2): Employed, Self-employed.

Sectors (84): JIP classification.

Year: 1970-2000.

Estimation of $ N_{ljt} $ , $ H_{ljt} $ , and $ W_{ljt} $ . Cross-Classified by Industry and Employment Status

First of all, we estimate the number of workers, hours worked and hourly wages cross-classified only by industry and employment status for each year. We combine several data sources such as the Population Census, Labor Force Survey, Manufacturing Census, Monthly Labor
Survey, Basic Survey on Wage Structure and others. Those estimates are adjusted to equal the sum of workers and income for employee with the estimates of the input-output table and the System of National Accounts. The opportunity cost of self-employed and family workers should be estimated. There are several alternative methods. We estimate it based on the ratio of marginal productivity between self-employed and employed workers which is derived from fitted values of the production function.

### Estimation of $ N_{kjt} $ , $ H_{kjt} $ , and $ W_{kjt} $ . Cross-Classified by More Detailed Category of Workers

The next step is to disaggregate our previous estimates to more detailed types of workers (gender, age, educational attainment). The estimation of the number of workers is based mainly on the Population Census. However, Japan’s census statistics do not report the detailed tables cross-classified. We estimate it from several related tables based on some assumptions.

The hours worked and hourly wages are estimated using the Basic Survey on Wage Structure and Monthly Labor Survey. These data are based mainly on the Monthly Labor Survey whose coverage is wider and more reliable. The Information of the Basic Survey on Wage Structure is used only as the difference ratios from average. We derived the wage rates of the self-employed from an estimation result of the production function.

## Estimation of the Sectoral Labor Input

The final step of our estimation of labor input is to estimate the Divisia index of price, quantity and quality for each sector. The total annual man-hour input of category l workers in industry j at time t is defined as the product of the number of workers and the average annual hours per worker:

$$ M H_{l j t}=N_{k j t}\;H_{k j t} $$

We define the growth of total real labor input in industry j at time t as a weighted average of the growth rates of man-hour input of all the categories.
$$ d\ln L_{j t}=\sum_{l}\overline{v}^{L}{}_{l j t}d\ln M H_{l j t} $$

where $ v^{L_{ljt}} $ with an upper bar denotes the average of the compensation shares of time t-1, $ v^{L_{ljt-1}} $ and the compensation shares of time t, $ v^{L_{ljt}} $ , $ v^{L_{ljt}} $ is defined by

$$ \nu^{L}{}_{l j t}=\frac{w_{l j t}M H_{l j t}}{\displaystyle\sum_{l}w_{l j t}M H_{l j t}} $$

We made some adjustment on $ MH_{jt} $ and $ w_{ljt} $ so that the total cost over all categories of workers in each industry is equal to the total value of labor compensation in that industry as given by the Input-Output table of the JIP Database.

We may now define an index of “quality of sectoral labor input,” or index of compositional change, as the ratio of labor input to working-hours:

$$ d\ln Q_{_{jt}}=d\ln L_{_{jt}}-\ln MH_{_{jt}} $$

where $ MH_{jt} $ is defined by

$$ M H_{j t}=\sum_{l}\ N_{k j t}\ H_{k j t} $$

A rising $ Q_{jt} $ means that the percentage of the higher paid categories in the workforce has increased in industry j over time.

#### A.3. Estimation of Annual Input-Output (IO) Tables

## Data Sources

Every five years, the relatively reliable linked IO table is available. Therefore we chose the years 1970, 75, 80, 85, 90, 95 and the final year 98 as our benchmark years. Major data sources for our annual IO tables for the benchmark years are

1970-1975-1980 Linked Input-Output Tables, Management and Coordination Agency;

1980-1985-1990 Linked Input-Output Tables, Management and Coordination Agency;
1985-1990-1995 Linked Input-Output Tables, Management and Coordination Agency;

1998 Extended Input-Output Tables, Research and Statistics Department, Economic and Industry Policy Bureau, Ministry of Economy, Trade and Industry.

For other years we used METI’s extended IO tables for every year.

## Compilation Process

Among the above IO tables, there are some differences in the rule of compilation and concepts. We adjusted these differences. The lease industry’s physical capital, which is rented to other industries, is treated as capital input in the lease industry. The cost of R&D in each sector is included in the production cost of that industry. The JIP Database is based on the 1968 SNA. Therefore, software investment is not included in investment. And depreciation of government capital is not included in the consumption expenditure of the government.

Next, we constructed converters to make adjustments for changes in industry classifications over time and aggregated the IO data into our 84 sectors.

We compiled IO tables in real terms (1990 prices) in the following way. 1970-1975-1980 IO tables contain real IO tables at 1980 prices. Similarly, 1980-1985-1990 IO tables contain real IO tables at 1980 prices. We linked these two real IO tables at year 1980. The second and the third IO statistics are linked at year 1990. The third and the fourth IO statistics are linked at year 1995.

The real values in linked IO tables are created by using price statistics such as the wholesale price index and the business service price index of the Bank of Japan in a way similar to the real values in the SNA statistics. Therefore, real values of output and intermediate input and implicit deflators in the JIP Database have basically similar characteristics as the corresponding SNA statistics of except for the treatment of the base year. Japan's long-term SNA statistics are based on a price vector of a single year. In the case of the JIP Database, real values and implicit deflators are created by linking real values of different base years.
### A4. Estimation of the Supplementary Tables

In the JIP Database we have also estimated the following supplementary tables.

(1) Trade Statistics by Industry and by Trade Partner Country: 1980, 1985, 1990, 1995, and 2000.

Using the supplementary converter table of the Input-Output Tables, Management and Coordination Agency; we converted the trade statistics of the HS 9-digit level, which are available at , into 302 manufacturing sectors, which are classified by economic activities. Using the Linked Input-Output Tables, Management and Coordination Agency, and wholesale price statistics of the Bank of Japan, we also calculated the trade statistics in constant 1990 prices.

(2) Inward and Outward Direct Investment and Service Trade Statistics by Industry

The data are based on Ito and Fukao (2003).

(3) Statistics on Japan’s Industrial Structure: Advertisement-Sales Ratio, Land Input per Worker, Herfindahl Index, Top 4-Frim Concentration Rate, Share of Firms Belong to Vertical and Horizontal Keiretsu Firms, etc.

The data are based on Ito and Fukao (2003).

(4) Sectoral Capacity Utilization Rate

For manufacturing and mining industries after 1973, we used the Index of Operating Ratio, Ministry of Economy, Trade and Industry, which is available at . For other industries before 1991 and manufacturing and mining industries before 1972, we employed the following estimation procedure. Following Burnside, Eichenbaum and Rebelo (1995) and Basu (1996), we assumed that the capacity utilization rate is closely correlated with the intermediate input-capital ratio. Following the “Wharton Method,” we linearly linked peak values of the intermediate input-capital ratio in each boom period and treated these interpolated