Science and Technology Indicators And A Catalog of Major S&T Indicators of Canada

Foad Shodjai Centre for Policy Research on Science and Technology (CPROST) Simon Fraser University Vancouver, BC, CANADA March 1996

Table of Contents


Since the World War II, and through the recent decades of the Cold War there has been a growing belief in the effectiveness of the forces of Science and Technology (S&T) in nation building. Government's chief mandate, briefly put, is the development of the nation with which it has been trusted---its economic, social, quality of life, etc. One major vehicle that is believed to achieve this end is Science and Technology (S&T). Programs in the governments that are to exploit this venue of developm nt are S&T Policy programs.

Chief activity in S&T change is Innovation. An understanding of Innovation process including its structure and sources, its functional and institutional players is required in order to promote, support, and sustain it.

The prime feedback to Governments' mechanism of policy-making is an extensive set of S&T Indicators. S&T Indicators help the policy maker bodies in a government, or non-governmental institutions in their advisory capacity, to evaluate past and present policies, and design and implement new ones.

The study of Indicators includes topics such as the very concept of indicators, it's models, input and output indicators, contributors and users of indicators, gathering, analysis, and reporting of indicators, standards and classifications, and their weaknesses and strengths.

This paper is a report on some aspects of my recent study of S&T Indicators. It is yet another step in my continued research into the issues of National S&T Policy in the framework of my graduate studies at the School of Communications, Simon Fraser University (SFU), Vancouver, Canada.

Science and Technology: A Mighty Force of Our Time

"The man of science is perceiving and endowed with vision, whereas he who is ignorant and neglectful of this development is blind. The investigating mind is attentive, alive; the callous and indifferent mind is deaf and dead. A scientific man is a true index and representative of humanity,..."


"few would dispute the claim that a nation's science and technology base is a critical element of its economic strength, political structure, and cultural validity."

---Eugene Garfield

The scientific advancements that humanity has achieved in the past two centuries---in hard and soft sciences, transportation, communication, agriculture, health and in many other fronts---out weights that of the entire recorded history that precedes it. For those who have been born in the past two or three decades, like the present writer, it is inconceivable to do economic analysis or policy without considering Science and Technology. Today, in the public's mind and many economists', philosophers' and politicians', S&T plays a core role in the development of our society, and in more than one aspect.

R&D, the most commonly accepted scientific activity, and its measurement are used by the Governments as a measure of their investment. The media reports and analyses R&D. Business magazines regularly publish data regarding R&D investment by firms (1). Stock Exchange reacts to R&D investment and planning. More and more, corporations are considering R&D data in their long term planning.

It is safe to conclude that in today's social structure Science and Technology have a pivotal role to play. Government officials, policy analysts, economists, and corporate Chief Executive Offices (CEO) closely follow Science and Technology changes, continuously evaluate its impact, and try to position their respective institutions such that it will benefit in form of what has become to be known as having the competitive edge.

Ignoring S&T is detrimental, be it by Governments, academics, or industry leaders. It is this powerful force that has claimed the attention of some of the best minds during the past few decades. Understanding the process of scientific and technological change, measuring it, promoting it, and using it to public's advantage (or one's own advantage) has been the subject of study and deliberation of a large number of researchers.

Of special interest in this paper is the field of Indicators. In order to discuss indicators, a few words will be said on models used in describing S&T change, economic model and its relationship with S&T, the nature of indicator itself and it's usage, innovation and R&D, and standards. In addition, a catalog of S&T indicators of Canada is produced, and a compendium of terms is maintained.

Science and Economics

Perhaps the most prevalent economical model used in conjunction with S&T is Douglas-Romer model(2). In this model economic development is directly dependant on capital, labour, and resources. To complete the picture knowledge is also added as a parameter. In a mathematician's language, economics is a function of capital, labour, resources, and knowledge:

E = Fn (Capital, labour, Resources, Knowledge)

This model indicates that increased knowledge would in turn result in economic prosperity---all other things equal, of course. Increasing knowledge, one would conclude, is a key issue to any economic policy, or in our case Science policy.

The first activity that increases scientific knowledge, and readily comes to one's mind, is Research and Development (R&D). R&D is held to be, by all the nations and the public eye, the most renowned activity that has increasing the scientific knowledge as its explicit goal. The truth of this statement and R&D's share of increased knowledge can be disputed.

A cursory study shows that there are other venues of increasing the pool of knowledge. The measurements of techniques employed by firms for acquisition, adaptation, and diffusion of new knowledge, or technology, is still at its early stage of development. However, lets consider the following two complimentary ways that are available to firms:

Purchase of knowledge. Scientific knowledge may be brought into a firm, an industry, or a country by purchasing, or licensing patents. Firms may also acquire new knowledge by non-market mechanism, for instance, searching public information systems (3).

Embedded knowledge. New knowledge can also be imported in form of embedded knowledge in machinery, methods, and procedures that are purchased. That is, the purchase of technology intensive capital goods will bring embedded knowledge to the buyer.

Therefore, knowledge can be expressed in terms of R&D, Purchased knowledge, and Embedded knowledge.

Knowledge = Fn (R&D, Purchased, Embedded)

In another word, there are more than one way of acquiring new knowledge. These may be thought of as input in an input/output model of knowledge generation:

The study and determination of the distribution of generated knowledge over these various sources of knowledge is one focal point of Science Policy. This sort of study requires modeling and a system of measurements. In this way, this field in itself is a user of S&T Indicators.

Innovation and R&D

A few words needs to be said about the relationship between Innovation and R&D. Often Innovation and R&D are used interchangeably. That is fine when the discussion is limited to R&D and its related issues such as indicators for R&D.

However, there is a distinction to be made which will help prevent misunderstanding along the way, and clarifies the scope of the studies, activities, and measurements that are performed and published. For most part, studies and reports produced by researchers are on R&D. Even those interested on the process and structure of innovation often find themselves directed, by people in the field, towards literature that is primarily about R&D and not innovation. This situation gives rise to a mental mpression that equates R&D and innovation. R&D is seen, by most people, to occupy the entire sphere of innovation activities. This impression is the point I like to identify and resolve in this section. Equating R&D and Innovation is a mistake that any student of S&T Policy must avoid.

But perhaps, the root of the problem, and at the same time its solution, should be traced to those documents where these terms are defined for standard use---in this case, I am referring to the Frascati Manual.

In pursuing the content of Frascati Manual, as a manual of standards for the Measurement of Scientific and Technical Activities, one might lose sight of the explicit limitations defined in the manual itself. Published methods and standards usually include definitions of their scope and premises. In the wake of the excitement generated in the pursue of these manuals, readers may neglect the limitations imposed on the discussion by the authors themselves.

The following passage from Frascati Manual clarifies the scope of its proposed standard practice:

"Innovation therefore involves a series of scientific, technological, organizational, financial, and commercial activities. R&D is only one of these activities."(4)

R&D, therefore, is subordinate to innovation, in that it is one activity which contributes directly to scientific and technological change. The implication is that in study of innovation one cannot be content solely with the issues of R&D. In fact, identification of other activities, their relationship and measurement becomes an integral part of innovation study.

Despite all said, the importance of R&D in structure of innovation cannot be denied. Wherever there are signs of R&D one can conclude that some degree of innovation must be taking place. However, not all innovative activities will necessarily have explicit R&D activities---at least in the sense defined by current standards. In another word, R&D maybe taken as gene marker (5) for innovation.


The concept of indicators in itself is an interesting field of science. Some things are easy to measure directly, such as the mass or speed of objects. Sometimes it is very difficult, or perhaps impossible, to directly measuring a phenomenon, such as quality of life, happiness, and of course, innovation. Indicators are used to tell us about things that are difficult to measure.

Technically, statistics are the building blocks from which indicators are constructed, and are based on the information available about the phenomenon under study. Often the two terms, statistics and indicators, are used interchangeably as is the case in the present paper.

As the literal meaning of the word indicates, indicators are indications of other things, usually each indicator emphasizing a specific aspect as well. For this reason, indicators are known to be indirect measurements of other things. As such, indicators are imperfect measurements, and therefore, several of them should be studied to see trends, a concept described as partially convergent indicators. By examining a set of indicators an analyst can discern trends in the behavior of the phenomenon t hand. Monitoring the fluctuation and movements of a number of indicators in groups over time reveals relationships between elements.

Indicators for science and technology may be defined as "statistics which measure quantifiable aspects of the creation, dissemination and application of science and technology. As indicators, they should help to describe the science and technology system, enabling better understanding of its structure, of the impact of policies and programs on it, and of the impact of science and technology on society and the economy." (6)

S&T data needs to be adjusted for the size of economy and labour they represent. Therefore, indicators are expressed in terms of ratio of two statistics---a specific numerator over a general denominator. For example, gross expenditure on research and development (GERD) over GDP. This particular indicator suggests that GERD is non-linearly related to GDP and that GERD/GDP for smaller economies is not to be expected to be close to the larger economies. (7)

Classification of Indicators

For us to start elaborating on a system of indicators as defined above, we need to start with some initial understanding of the structure of science and technology. For instance, one would expect that performing S&T activities would cost, and therefore, there are funding institutions, those that provide monetory resources for S&T activities, and there are performing institutions, those that perform the S&T activities. So, it would be natural to want to know who funds who, how much, and in w at ways. Who performs the activities, and how it is distributed among the players.

We also know that the nature of S&T activities vary based on the aims of the performing institutions, which hints to classification of S&T activities based on sector, industry, or field of science. Therefore, it would be desirable to know the measure of S&T activities for various fields of science and technology.

It is also known that a whole range of people are involved in S&T activities, such as researchers, professors at universities, and technicians. How many people are involved? What are their qualifications? What is the distribution of this force by sector? by performing institutions?

The point being that an initial understanding of the structure and sources must be in place.

It is suggested here that an initial perspective of the structure and source of S&T existed prior to establishing the current system of S&T Indicators and its standard. This initial structure, although not explicit in the literature, can be seen in the manuals for standard practices and survey that are widely used. Further, it is observed that since the inception of these standards, some 30 years ago, no significant changes have been made to the structure of S&T Indicators. The continuous r vision and updates to manuals of standards, such as Frascati, have been in response to a need for international comparison, rather to mark a change in the perception of S&T itself. This static state may suggest that the underlying perception of S&T's structure and source is not changed, or at least not significantly changed, from the initial one.

The Most Common Units of Measurement

No matter what structure, or model, of S&T activity is adopted, it is almost certainly true that the measurement of the input to the model is much easier than it's output. In cases where output is measured, its correlation with input is a difficult task, specially because the output is "the result of a combination of many inputs, of which scientific research is only one." (8)

In the most prevalent structures of S&T activities, the units of measurements are money and people. These are of course input to the system. Measuring in terms of money and people has it's strength and weaknesses.

Money is a useful measure because many other areas such as economic and social indicators are also expressed in monetary units. Therefore, monetary measurement of S&T activities allows a comparison of investment on S&T and these other areas, specially when the statistical units are the same or closely match each other. Monetary data over an extended period of time shows trends in expenditure on S&T activities. The availability of monetary data, due to financial requirements of both funding nd performing institutions, is another advantage of this form of measurement.

Many R&D policies are determined through the control of financial resources available to it, specially by the governments. In this sense monetary measurements play an important role in providing trends in policies and scientific activities.

The expectation of each monetary unit to have the same effect as the other; and the exchange rate for national comparison and over time are among the constraints of monetary measurement.

Measuring people is more problematic than the monetary one. Ease of head count, and its usefulness in determining supply and demand are its advantages. However, no person spends its entire day on S&T activity, specially if working on multiple projects. Therefore, the concept of full-time equivalent person-years (PYs) has been used to obtain per capita data. Differences in a normal working day from sector to sector and from country to country is yet another complicating factor in comparison o data related to human resources involved in R&D activities.

People involved in S&T activity are of various qualifications and act in widely different capacities and roles. For instance, highly qualified scientists contribute in a different capacity than laboratory technicians do. This situation requires a classification of type of activities and levels of qualifications. And lastly, just as the monetary measurement, there is an expectation that each human unit has the same effect as another in the same category.

Sources, Users and Collectors of S&T Indicators

In any system of indicators there are three parties who need to collaborate for the system to be successful. First party includes those who are considered to be the source of data, i.e., those who contribute to the statistics, such as the funding institutions, and the performers. They are usually referred to as statistical units. One classification of statistical units for input to R&D activities, and a very detailed one, is to be found in OECD's Frascati Manual.

Sources of data may be categorized into three major groups (9). One, the individual, who provides the following:

· Information on the characteristics of the people involved in S&T activities
· Time spent in S&T activities, and
· Public awareness and understanding of S&T issues.

The second group comprises institutions financially supporting S&T activities, which includes private non-profit organizations, government granting bodies, and public and private enterprises awarding contracts for S&T activities. This group is the source of the following data:

· Magnitude of fund available for S&T activity in a particular economy or country
· Distribution of the total fund by donor type
· and, S&T activity performed in the external sector

The third group comprises institutions in which the S&T activity is performed. This is perhaps the most important source of data. This group consists of four major sectors: Government, business, private non-profit, and higher education. Each sector has its own challenges in collecting the S&T data. The business sector is composed of firms that are of different size and economic activities, therefore a number of different surveys may be required. Larger firms are well established and have report ng procedures implemented for their various S&T activities. However, in cases where they are involved with multiple S&T activities, it is difficult to report the distribution of the resources over the activities accurately. Smaller firms however are many in number but only a few are engaged in S&T activities, and therefore sampling would be misleading. Small S&T performing firms maybe identified through the data from the second group, i.e., the funding institutions, in order to make a bett r sampling group.

In the government sector, the number of groups involved in S&T activities may be far less than the business sector. They also may be at different levels of government such as local, regional, and national, and with different mandate such as defense or education. Governments are the largest single source of fund for S&T activities. Due to their accountability to the public, governments usually have a well established accounting system which make the data more readily available.

The private non-profit sector is largely composed of funding institutions. It includes foundations, health-oriented organizations.

The last sector, higher education, represents the most difficult source of data in terms of its collection. The reason for this difficulty may be found in the fact that people in this sector have a wide range of activities and their sources of funding are many. S&T activities may be part of the teaching functions. Students, as they progress in their research, may also be involved in S&T activities. Estimation plays a greater role in collecting data from this sector than any other sector.

The second player is the user of the data. Included here would be governments at local, regional, and national levels, policy analyst, and most recently large corporations. These is no special classification of users beyond grouping them as public and private users. Agencies or units that provide advice to government control or decision-making bodies on S&T and/or economic policy, agencies with large S&T operation or funding responsibilities, and the legislature---these may be cited under the public sector. Anyone else interested in influencing the public sector would fall under private sector!

The third and final player is the collector of the data. The collection of data is a very involve process. In the following excerpt, H. Stead, outlines some of the activities of the collectors of S&T data.

"The needs of actual and potential clients must be understood and defined. Sources of information must be identified, contacted and persuaded to co-operate. The data available must be evaluated for relevance and accuracy. The co-operative development of information needed but not currently available must be systematically undertaken. Resources for data collection and analysis must be allocated and the statistical program developed accordingly. Data collection methods and instruments must be prep red and tested." (10)

There is very little said about the collector of S&T data in the literature. The study of it's infrastructure, at local, regional, and national level; identification and establishment of the information technology (IT) required to support and facilitate activities for an accurate and speedy collection of S&T data---are but a few tasks in an enterprise that has the placement, or enhancement, of a national system of S&T indicators as its goal.

Sources of Error

Each sector as a source of S&T data has its own challenges in identifying, collecting, and reporting its S&T activities. In general the lack of proper records by S&T activity is the primary source of problems in producing accurate data. This problem is most acute in the higher education institutions. Sources of S&T statistical data, specially business and higher education sectors, do not have an appropriate accounting system based on S&T activities, and therefore, the financi l expenditures and human resources must be estimated. It is this level of estimation required which is the main source of concern.

"It is the degree of estimation necessary which distinguishes the reliability of S&T statistics from the traditional economic and social statistics based on established accounting classifications. In some cases, S&T surveys are replaced or supplemented by administrative records such as those of S&T contracts and grants, tax returns, and national registers of professional engineers." (11)

Manuals of International Standards

There are sets of documents that put forth international standard practice and survey of S&T related activities. Notably among these sets are two, one by United Nations Educational, Scientific and Cultural Organization (UNESCO), the other by Organizations for Economic Co-operation and Development (OECD).

The UNESCO and OECD documents are compatible, yet different in that OECD's set, although within the UNESCO's recommendations, is specific to R&D and to the needs of OECD member countries (for a list of OECD member countries see the Compendium). Also, OECD considers S&T activities only as they relate to R&D.


The UNESCO Division of Statistics on Science and technology has organized the systematic collection, analysis, publication and standardization of data concerning science and technology since 1965. It primarily concerns itself with human and financial inputs to R&D. (12)

UNESCO's achievements in the development of measuring scientific and technical activities are marked by the following publications:

· Provisional Guide to the Collection of Science Statistics
· Manual for Surveying national Scientific and Technological Potential
· Guide to the Collection of Statistics on Science and Technology
· Recommendation Concerning the International Standardization of Statistics on Science and Technology

In the above set, the last one is a response to the need expressed for international standards that could be used and applied by all member states.


The OECD document set includes the following:

· Proposed Standard Practice for Survey of Research and Experimental Development (Frascati Manual), 1992
· Manual for the measurement and interpretation of Technological Balance of Payment Data, 1990
· OECD Proposed Guidelines for Collecting and Interpreting Technological Innovation Data (Oslo Manual), 1992
· Practical Guidelines for the Use and Interpretation of Patent Data as Science and Technology Indicators
· Recommendations for the Development of Indicators of Human Potential in Science and technology

These documents are continuously updated by OECD. Frascati Manual is the counterpart to the item emphasized in the UNESCO's set (i.e., the last item).

Frascati Manual

No paper on indicators related to science and technological change can be complete without mentioning the Frascati Manual, or its sister document, Oslo Manual.

With its first draft some 34 years ago, Frascati Manual achieves its goal in defining a standard for practice, classification and purpose of R&D input data. It is widely used and frequently sited in papers like this one.

Frascati fits within the recommendations of UNESCO, at the same time its specific to the needs of OECD member countries. Member countries have been collecting data related to S&T since 1960s. However, international comparison was impossible in presence of the differences in concepts, methods, and scope of the data collected. Frascati is a result of the need to standardize the data collected, and its reporting.

The practice described in this Manual has brought into light many aspects of innovation, thereby contributing significantly to the growing knowledge of the structure and sources of innovation. A side effect of this knowledge is a realization that data exclusively on R&D, and in particular limited to "input," is not a true measure of the subject at hand. Although necessary, R&D input data needs to be augmented by other types of data to form a better perspective of innovation. There are therefore several points to bear in mind when considering this Manual:

R&D input data. It exclusively deals with R&D input data:

"This Manual [Frascati],... deals exclusively with the measurement of human and financial resources devoted to Research and Experimental Development (R&D) often referred to as R&D `input' data" (13)

Government fiscal incentives. In it's measurement of expenditures devoted to R&D it does not include government's fiscal incentives which are offered in terms of tax credits for R&D performing firms. This fiscal incentive is government's tax revenue forgone for promoting R&D (14). In case of Canada, the tax credit amounts to a substantial source of funding for R&D performing firms.

Linear model. Embedded in the approach adopted by Frascati Manual in its classification and description of indicators is a an assumption based on a linear model. It assumes a linear model in growth of scientific knowledge. Briefly, the linear model advocates that innovation takes place in a linear fashion: it starts with research, then invention, moving onto innovation, and finally diffusion of new techniques. Linear model, though widely used, is increasingly believed to be too abstract to adequ tely represent the dynamics of S&T change. Nevertheless, it continues to inform policy makers. For this reason the economic historian Nathan Rosenberg (1991) said the linear model "is dead, but it won't lie down." (15)

There are other models used in describing systems of innovation---the Input/Output model, the Interactions between Opportunities, Capabilities, and Strategies, the Chain-Link, and Neural Network models. The three latter models take a much more sophisticated approach in describing the structure of S&T change than the Linear model (16).

Uniformity of structures and resources. Frascati Manual assumes a uniformity of structure and resources of innovation across industries and countries. Such uniformity does not exist. In fact, ever increasingly, the unfoldment of a perception which the R&D data is bring about suggests a marked difference in the process of innovation in industries and countries in both structure and sources.

Another aspect of this assumption based on uniformity is related to the degree of effectiveness of measured units in the system. For example, it assumes that every dollar spent in one industry or country will have an equal effect as a dollar spent in another industry, or country. The same holds for human resources measured across industries and countries.

In summary, in pursuing the content of Frascati Manual, and employing its proposed standards in creating a national system of R&D Indicators, one has to be conscientious of its explicit scope and implicit assumptions and limitations a few of which was briefly outlined above.

S&T Indicators in Canada

Canada is a member country of OECD. It's concept, definition and methodology for collection of statistics on research and development is based on OECD's guidelines with some adaptation to the Canadian situation.

Grouping on an input-vs.-output basis the framework of Canada's system of indicators, the measurements of people and money involved in Canada's S&T activities are indicators on the input side, while economic return, patent and citation data are on the output side.

R&D Performers and Funders Categorized

This section only outlines the categorization of Canadian R&D players as found in reports. For detailed definition and a discussion see Frascati Manual and A Framework for Measuring Research and Development Expenditures in Canada by Statistics Canada.

The purpose of this listing is primarily for displaying the entire categorization at several levels of detail. It helps building a bird's eye view of the landscape of Canadian players as viewed by reporting agencies, specially that of Statistics Canada.

1. Business Enterprise
	Agriculture, fishing and logging
	Mines and Wells
		Services incidental to mining
		Crude petroleum and natural gas
		Beverages and tobacco
		Rubber products
		Plastic products
		Textiles Wood
		Furniture and fixtures
		Paper and allied products
		Printing and publishing
		Primary metals (ferrous)
		Primary metals (non-ferrous)
		Fabricated metal products
		Aircraft and parts
		Motor vehicles, parts and accessories
		Other transportation equipment
		Telecommunication equipment
		Electronic parts and components
		Other electronic equipment
		Business machines
		Other electrical products
		Non-metallic mineral products
		Refined petroleum and coal products
		Pharmaceutical and medicine
		Other chemical products
		Scientific and professional equipment
		Other manufacturing industries
		Electrical power
		Other utilities
		Transportation and storage
		Wholesale trade
		Retail trade
		Finance, insurance and real estate
		Computer and related services
		Engineering and scientific services
		Management consulting services
		Other services
2. Private Non-profit Organizations
	Private philanthropic foundations
	Voluntary health organizations
	Societies and associations
	Research institutes
3. Government
	Federal Government
		Agriculture and Agri-Food Canada (AgCan)
		Atomic Energy of Canada Limited (AECL)
		Canadian International Development Agency (CIDA)
		Canadian Space Agency (CSA)
		Environment Canada (EnvCan)
		Fisheries and Oceans Canada (F&O)
		Health and Welfare Canada (Health Canada) (HWC)
		International Development Research Centre (IDRC)
		Industry Canada (IndCan)
		Medical Research Council of Canada (MRC)
		National Defense (NDEF)
		National Museums of Canada (NMC)
		National Resources Canada (NRCan)
		National Research Council Canada (NRC)
		Natural Sciences and Engineering Research Council Canada (NSERC)
		Social Science and Humanities Research Council of Canada (SSHRC)
		Statistics Canada (StatCan)
		Others Departments
	Provincial Governments
		Prince Edward Islands
		Nova Scotia
		New Brunswick
		British Columbia
		North West Territories
4. Higher Education
5. Abroad

Current Indicators

Current indicators of Canada's scientific and technological activities are: (17)

· expenditures on research and development
· federal government scientific activities
· personnel working in science and technology
· Canadian research output (citations)
· Canadian patented inventions
· international payments and receipts for technology
· trade in selected commodities

The first five activities are cataloged in this section. A brief description and notes on their usage, calculation, strength, and weaknesses are included. A selected type of charts or reports found under each activity is also listed.

1. Expenditures on Research and Development

The standard measuring indicator of S&T expenditure is the summary statistic, gross domestic expenditures on R&D or GERD. For the most part the Canadian GERD is compatible with OECD definitions, but differences do exist. OECD considers both the Natural Sciences and Engineering (NSE) and the Social Sciences and Humanities (SSH) when measuring GERD. Canada, however, only includes NSE. Statistics on SSH does exist, and the OECD version of Canadian GERD is presented in two forms: NSE alone, and NSE and SSH together.

GERD is reported as a matrix of funding and performing sectors. Such a matrix essentially measures the flow of R&D fund from one unit to another. The matrix is illustrated below.

Table 1: GERD Matrix

  Funding Sector                Performing Sector                     Total by 
                    ----------------------------------------------    Funding
                    Business    Private     Government   Higher       Sector
                    Enterprise  non-Profit               Education               
     Business      |           |           |           |          |
    Enterprise     |           |           |           |          |
                   |           |           |           |          |
Private non-profit |           |           |           |          |
                   |           |           |           |          |
    Government     |           |           |           |          |
                   |           |           |           |          |
    Public GUF     |           |           |           |          |
                   |           |           |           |          |
 Higher Education  |           |           |           |          |
                   |           |           |           |          |
      Abroad       |           |           |           |          |
     Total by      |           |           |           |          |
 Performing Sector |           |           |           |          | National GERD

The following paragraphs outline major characteristics of GERD and its usage in R&D measurement.

Usage: The GERD is an indicator of S&T activities. It is appropriately used as a summery of R&D activities and the basic flow of funds. It serves as a general indicator of S&T activity and not as a detailed inventory of R&D projects within an organization, sector, or country. It is an estimate and as such can show trends in R&D expenditures by sector and sub-sector, by region and country, from year to year. In this capacity, the GERD estimates are sufficiently reliable for th ir main use as an aggregate indicator for science policy.

Calculation: Sum of both Intramural and Extramural expenditures. Procedures for measuring R&D expenditures, as listed in the Frascati Manual, are:

· identify the intramural expenditures on R&D performed by each statistical units;
· identify the sources of the funds for these intramural R&D expenditures as reported by the performers;
· identify the extramural R&D expenditures of each statistical units; and
· aggregate the data, by sector of performance and source of funds, to derive national totals.

There are two criterion to include the fund as R&D flow:

· there must be a direct transfer of resources;
· and, this transfer must be both intended and used for the performance of R&D.

Strength: As a single aggregate indicator it shows trends in R&D expenditure by sector and sub-sector, by region and country, from year to year. In this capacity, the GERD estimates are sufficiently reliable for their main use as an aggregate indicator for science policy.


· Fluctuating dollar values, specially in case of intramural expenditures
· It is only approximately true
· One of the most important problems relating to GERD concerns it's definition. There remains a certain level of ambiguity in defining what constitutes R&D activities.

Type of data reported:

· Canadian R&D Spending Flow
· Expenditures on R&D, by Performing and Funding Sectors
· R&D in Canada---Who Funds, Who Performs
· Federal S&T Expenditures
· National R&D: 1. Actual Dollars, 2. Real Growth Percentage, 3. GERD/GDP
· Intramural R&D Expenditures, by Industry: 1. R&D, 2. Average R&D/Sale Percentage
· Industrial R&D, by Source of Fund, and Selected Sectors
· Top 50 Companies in R&D Expenditures: 1. R&D Spending, 2. Percentage of change, 3. R&D Revenue
· Investment Tax Credit for R&D. 1. Amount claimed, 2. Number of taxpayer Claims in the reporting period
· Regional Expenditures

2. Federal Government Scientific Activities

The Federal Government is highly involved in S&T activities as both funder and performer. The Federal Government is the largest funding institution in Canada ($6 billion in 1993-4) with a generous Tax Incentive program ($1 billon in 1993-4) totaling $7 billion in 1993-4. Half of the expenditure (i.e., $3.1 billion) was on intramural S&T activities which places the Federal Government as one of the major performer as well.

Therefore, both monetary and personnel data are reported on the Federal Government as funder and performer of S&T activities.

Type of data reported:

· Federal S&T expenditure over years
· Federal S&T expenditures, by Federal department and agencies
· Federal S&T expenditures, by performing sector, over years
· Total Federal expenditures on S&T, by activity (extramural, contracts, grants, fellowships)
· Federal R&D and S&T expenditures, by area of application
· Federal personnel engaged in S&T activities, by major departments or agency
· Federal Personnel engaged in S&T activities, by category (functional)
· Regional distribution of Federal R&D and S&T expenditures: by region, by performer, and by Province
· Federal Natural Sciences and Engineering facilities, by Province and department

3. Personnel Working in Science and Technology

The following functional categorization of personnel engaged in R&D activities are found in statistical reports:

1. Executive
2. Scientific and Professional
3. Administration and Foreign Service
4. Technical
5. Administrative Support
6. Operational
7. Military

The following occupational breakdown of the personnel in R&D activities are also used:

1. Managers, Administrators, and Related
2. Natural Scientists
3. Physical Scientists
4. Life Scientists
5. Architects and Engineers
6. Mathematicians, Statisticians, and Systems Analysts
7. Social Scientists
8. Teachers
9. University Teachers
10. Medicine and Health (includes first professional degrees, M.D., D.D.S., D.M.V., with masters' and doctoral degrees)
11. Sales Occupations
12. Services Occupations

The most widely used indicator on personnel working in S&T is the Highly Qualified Personnel (HQP). What follows describes some characteristics of this indicator.

Usage: Head count is used in determining supply and demand and in planning training and hiring. Person-year is used to calculate per capita expenditures.

Calculation: There are two widely used indicators:

· R&D scientists and engineers per 1,000 labour force; and
· all scientists and engineers per 1,000 labour force

Strength: The strength of HQP data lies in its usage, i.e., it helps determining supply and demand, and planning training and hiring.

Weakness: Head counts and full-time equivalent data assumes a uniform effect of people engaged in R&D.

Types of data reported:

· HQP per 1,000 labour force
· Percentage of scientists engaged in R&D
· Distribution of labour force by level of education attainment---occupational
· Distribution of personnel engaged in S&T activities by category---functional
· Graduate Degrees by field of study,
· Foreign students as a percentage of full-time graduate enrolment, by fields of study

4. Canadian Research Output (Citations)

Canada also collects and reports on output of R&D activities. One such indicator is measurements of scientific literature in form of publications and citations. One series of statistics counts number of publications. It is usually reported by fields of research. Another series counts citation of Canadian publications internationally, again reported by fields of research. Both of these series are usually associated with the university sector where publication is the only method of dissemination, and therefore, a direct indicator.

Usage: One main usage is for international comparison. It also helps identifying most-cited authors, high impact papers and journals, leading university and corporations, and hot subject areas.

Calculation: There are national and international databases where publications are registered. Data collected includes date, author, institutions, subject matter, and citation. One such prominent database is the Science Citation Index (SCI) database which is developed and maintained by the Institute for Scientific Information (ISI). ISI's index includes 15,000,000 papers published since 1945 and more that 200,000,000 citations.

Strength: For the university sector citation analysis is a direct measurement.

Weakness: Citation is a direct indicator for university sector, but it must be used with caution in the Government and industry. Scientists working for the Government and consultants in the industry disseminate their work through other means and often do not receive proper acknowledgment for their work. Another problem in this series of indicator is to be found where corporate secrecy prevents publication of scientists' work in the industry. There are a few other issues of concern with citation ata: (18)

· One related to whether citations represent agreement or disagreement. In hard sciences, citation generally means agreement, whereas in social sciences critical citation is more common. Therefore, raw citation data may not be an indicative of an author's contribution to the field and its context and content must be examined as well.
· Self-citation is another question raised in connection with citation data. Self-citation is an accepted practice because science is viewed as cumulative process. However, excessive self-citation may distort the citation analysis. In any event, excessive self-citation may be identified and adjusted.
· Citation circles are another expression of self-citation phenomenon. This may inflate the group`s ranking in a field. However, such conspiracy requires a substantial number of publications by the group.
· There is a concern that methods tend to be cited more frequently than theoretical papers. This perception is not supported by studies performed by the Institute for Scientific Information's (ISI).
· Obliteration should also be considered. It refers to the fact that breakthrough achievements, such as Einstein's theory of relativity, are cited less and less over time.

Type of data reported:

· National shares of World publication output, by selected countries (count, and % changed)
· National shares of Citation, by selected countries (count, and % changed)
· Canadian output by field (publications and citations data)

5. Canadian Patented Inventions

Usage/Strength: Patent is a good measure of accumulation of national intellectual capital. It represents one aspect of country's R&D effort. Its a good approximation for technological sophistication.

Calculation: Patent office records are used to determine the figures. Canadian Intellectual Property Office (CIPO) maintains such records. CIPO received 26,865 applications in 1993-4, granted 17,247 patents.


· Individual patents are of different values. One patent may represent a major breakthrough in technology and responsible for creation of a new market; while another may have a much more limited effect.
· Many important inventions are not patented at all. The complexity and difficulties of obtaining patent, and corporate secrecy are the main reasons.
· Patent is a measurement of invention and not innovation.
· Not all application and granted patents are domestic. Figures need to be adjusted to show an accurate domestic and foreign data.
· Many corporations patent the same invention in multiple countries. This fact needs to be considered in international comparison.
· Differences of criterion in countries for granting patents make international comparison more difficult.
· Canadian residents file more patent in the U.S. than in Canada. Perhaps the advantages that a larger U.S. economy offers accounts for this.

Type of data reported:

· Domestic patents per 1,000 labour force.
· External patents. It represents corporation's assessment of a country's to protect, use, and market a technology


Indicators are based on the underlying statistical data on R&D activities. The classification, collection, and interpretation of R&D data reflect a perception of the structure, resources, activities, and objectives of innovation. This perception must need to evolve as the former set of studies bring about a better understanding of innovation. At this time, after three or four decades, we should pause and redefine our perception, and consequently update the standards for survey and classificatio of innovation studies to reflect a new and dynamic framework.

A few observations are already made in the recent literature:

Three major results may be observed from the total experience of Innovation studies. First, innovation has a wide industrial distribution. It is not confined to large multi-national high-tech firms. Innovation in Small and Medium-sized Enterprises (SME) is pervasive. Total sum of SME's part-time personnel in innovative activities are comparable with large firms' and full-time personnel. Second, it follows that R&D is not necessarily a good indicator of innovation intensity. R&D maybe taken as " ene-mark" for innovation. Where R&D is found there is probably some innovation taking place, but the exact measure and intensity cannot be determined by R&D alone. In addition, absence of R&D does not necessarily indicate absence of innovative activities. Thirdly, there is considerable variation in the structure, sources, and purpose of innovation among industries. This observation implies that industry specific studies are required if we are to build an accurate picture.

In general, innovation is far more pervasive than originally perceived, and far more variable in its structure, resources, activities and outcomes than recognized by current policy instruments, methods, and standards (19).

This diversity exists among countries, but more importantly within national bounders---within and between the institutions in industries and therefore intra-industry diversity. A system of policy instruments and methods will have to be informed of the structures and sources of innovation and its diversity in order to design and enact effective policy measures. Therefore policy will have to depend on a better understanding of innovational complexity and variations, a process which has hardly begun to be explored in empirical terms.

Compendium of Related Terminologies and Definitions, Version 2.0

Notes on Version 2.0

This compendium started in my previous study. This new listing now has more than 120 new entries, bringing the total to just undser 200 (some entries have multiple items). As it is growing, the entries seem to fall into a few distinct categories, such as:

· Acronyms and Abbreviations
· Definition of standards for survey and practice
· Institutions and Agencies (government and private)
· Papers and Articles
· People and Biographies
· Studies (research and surveys)

The future versions of this compendium will group the entries under appropriate headings, and will be based on a searchable database provided over the Internet with a Web Browser interface.

Notes on Version 1.0

There are many working definitions and acronyms found in the core manuals on Innovation and Indicators. As the subject of my thesis will be very closely related to it, I decided to start compiling a glossary of terms covering acronyms and definitions and major references for each one. Such a compendium is essential to help finding the exact definition and reference of various terminologies quickly. As such this collection is continuously maintained and essential to my research in the immediate futu e.

Entries are alphabetically ordered. Major references are also included for each definition or description. References to Frascati and Oslo Manual are paragraph numbers (see bibliography for which revision is used).


Acquisition of disembodied technology

Acquisition of embodied technology




Applied Research



Basic Research



Binary data type


Capital Expenditures



CHI Research Inc.












Diffusion (of innovation)





Enterprise-type unit



Experimental Development

Extramural R&D Expenditure













High-Technology Industries












Innovation, R&D and Scientific and Technological

Innovation Surveys

Innovation, technological

Innovative activities

Intangible investment

Innovation System


Intramural R&D Expenditure











Manufacturing start-up and pre-production development

Marketing for new products
























Ordinal data type



Patents, Domestic

Patent, External

Partially Convergent Indicators



Political nature of science and indicators (paper))



Product Innovation

Process Innovation

Product Differentiation



R&D Personnel

R D & D

Researchers (RSE)


REP for field A in country = (%share of field A in Country X) / (%share of field A for all 8 Countries)



Science Watch

Sector, Abroad

Sector, Business Enterprise

Sector, Government

Sector, Higher Education

Sector, Private Non-Profit

Significant R&D Activities














Supporting Staff, Other





Technicians and Equivalent Staff


Tooling-up and industrial engineering








Selected Bibliography

Government of Canada, Resource Book for Science and Technology Consultations, 2 Vols., Secretariat for Science and Technology Review, Industry Canada, June and August 1994.

Lipsett, Morely S. and Smith, Richard K. "Cybernetics, and (real) National Innovation Systems," School of Communications, and Center for Policy Research on Science and Technology (CPROST), Simon Fraser University. Paper prepared for 1995 IEEEE International Conference on Systems, Man, and Cybernetics, October 22-25, 1995 Vancouver, British Columbia, Canada.

Lipsett, Morley S., and Holbrook, A., and Lipsey, Richard G. "R&D and Innovation at the Firm Level: Improving the S&T Policy Information Base," CPROST, Report CP 95-9. Paper presented at the Fourth International Conference on S&T Indicators, Antwerp, October, 1995.

OECD, Frascati Manual 1992: Proposed Standard Practice for Survey of Research and Experimental Development, September 17, 1992.

------, Proposed Guidelines for Collecting and Interpreting Technological innovation Data (Oslo Manual), September 12, 1991.

------, Development of S&T Output Indicators, Contribution by Industry, Science and Technology Canada, Group of National Experts on Science and Technology Indicators, Committee for Scientific and Technological Policy, drafted April 26, 1993, Paris.

------, Regional R&D Indicators, Group of National Experts on Science and Technology Indicators, Committee for Scientific and Technological Policy, drafted April 20, 1993, Paris.

------, Statistics and Indicators for Innovation and Technology, and Annex 1 & 2, Working Group on Innovation and Technology Policy, Committee for Scientific and Technological Policy, distributed April 19, 1994, Paris.

Science and Public Policy, Journal of the International Science and Policy Foundation, Vol. 19, No. 5 October 1992, and No. 6 December 1992, Great Britain: Beech Tree Publishing, 1992.

Statistics Canada, A Framework for Measuring Research and Development Expenditures in Canada, Catalogue 88-506E, Ottawa: 1984.

Statistics Canada, Indicators of Science and Technology 1990---Resources for research and Development in Canada nd Technological Balance of Payments, Catalogue 88-002, Vol. 1, No. 3, Ottawa: 1990, and Vol. 2, No. 4, Ottawa: 1992.


Adam Holbrook, Science and Public Policy, Vol. 19, number 5, p. 262
A. Holbrook, Science and Public Policy, October 1992, p. 270
Frascati, paragraph 20
A term used by Adam Holbrook in our conversations in Fall of 1995.
Martin B. Wilk, Statistics Canada, A Framework for Measuring Research and Development Expenditures in Canada, catalogue 88-506E, preface.
A. Holbrook, Science and Public Policy, Vol. 19, No. 5, p. 264
A. Holbrook, Science and Policy, Vol 19, No. 5, p. 264
Based on NAME, "article," Science and Public Policy, Vol. 19, No. 5. pp?-?
H. Stead, Science and Public Policy, Vol 19, No. 5, p. 275
H. Stead, Science and Public Policy, Vol. 19, No. 5, p. 279
Based on Statistics Canada publication, A Framework for Measuring Research and Development Expenditures in Canada, Appendix I, 88-805E
Frascati, paragraph 3
M. Lipsett, <ref?>
John de la Mothe, Resource Book, Vol II, p. 12
Use of models in Innovation studies has been cover by the present writer in a paper titled Systems of Innovation, 1995.
This chapter is mostly based on the following publications of Statistics Canada: A Framework for Measuring Research and Development Expenditures in Canada (88-506E); Resource Book for Science and Technology Consultation, 2 Vols.; Selected Science and Technology Statistics 1993.
Science and Public Policy, Vol. 19, No. 5, pp. 325-6
There is more than one instance where researchers have concluded the pervasiveness of innovation, see Lipsett & Smith, "Cybernetics..." and Keith Smith, "Technological Innovation Indicators: Experience and Prospects", Science and Public Policy, Vol. 19, No. 6, p. 390

* * *

Foad Shodjai
Centre for Policy Research on Science and Technology (CPROST)
Simon Fraser University
Vancouver, BC, CANADA