| Title: | Download and Measure Global Trends Through Google Search Volumes |
|---|---|
| Description: | Google offers public access to global search volumes from its search engine through the Google Trends portal. The package downloads these search volumes provided by Google Trends and uses them to measure and analyze the distribution of search scores across countries or within countries. The package allows researchers and analysts to use these search scores to investigate global trends based on patterns within these scores. This offers insights such as degree of internationalization of firms and organizations or dissemination of political, social, or technological trends across the globe or within single countries. An outline of the package's methodological foundations and potential applications is available as a working paper: <https://www.ssrn.com/abstract=3969013>. |
| Authors: | Harald Puhr [aut, cre] (ORCID: <https://orcid.org/0000-0002-3308-9553>), Jakob Muellner [ccp] (ORCID: <https://orcid.org/0000-0002-3443-0469>) |
| Maintainer: | Harald Puhr <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.0 |
| Built: | 2026-05-25 07:35:43 UTC |
| Source: | https://github.com/ha-pu/globaltrends |
The function adds one or more batches of keywords with a time period for downloads to the database. The batches serve as input for all download and computation functions.
add_control_keyword(keyword, start_date = "2010-01", end_date = "2020-12") add_object_keyword(keyword, start_date = "2010-01", end_date = "2020-12")add_control_keyword(keyword, start_date = "2010-01", end_date = "2020-12") add_object_keyword(keyword, start_date = "2010-01", end_date = "2020-12")
keyword |
Keywords that should be added as batch. Vector of type
|
start_date |
Start of time frame for which the batch data should be downloaded. Object
of type |
end_date |
End of time frame for which the batch data should be downloaded. Object
of type |
Since Google Trends allows a maximum of five keywords for each query, batches
of control keywords can consist of up to five keywords. Since one control
keyword is added to batches of object keywords for mapping, object batch
length is limited to four keywords. When a character vector contains
more than four (five) keywords, the vector is split into four-keyword
(five-keyword) batches. A list must contain character vectors
of length four (five) or less. Each batch of keywords is combined with a time
period for which data will be downloaded. To change the time period for an
existing batch, all downloads and computations must be rerun.
Message that the batch has been created successfully. Batch data is written to tables batch_keywords and batch_time. Numeric vector containing the newly added batch numbers are returned.
If you use search topics for object keywords, make sure to use search topics for control keywords and vice versa. See Google's FAQ for additional information on search topics.
To avoid trailing spaces stringr::str_squish is automatically
applied to all keywords.
## Not run: add_control_keyword( keyword = c("gmail", "maps", "translate", "wikipedia", "youtube"), start_date = "2016-01", end_date = "2019-12" ) add_object_keyword( keyword = c("apple", "facebook", "google", "microsoft"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = c("gmail", "maps", "news", "translate", "weather", "wikipedia", "youtube"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = c("amazon", "apple", "facebook", "google", "microsoft", "netflix", "twitter"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = list( c("gmail", "maps", "news"), c("translate", "weather", "wikipedia", "youtube") ), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = list( c("amazon", "apple", "facebook", "google"), c("microsoft", "netflix", "twitter") ), start_date = "2016-01", end_date = "2019-12" ) # search topics add_control_keyword( keyword = c("%2Fm%2F02q_bk", "%2Fm%2F055t58", "%2Fm%2F025sndk", "%2Fm%2F0d07ph", "%2Fm%2F09jcvs"), start_date = "2016-01", end_date = "2019-12" ) # This adds the following topics: Gmail, Google Maps, Google Translate, Wikipedia, YouTube ## End(Not run)## Not run: add_control_keyword( keyword = c("gmail", "maps", "translate", "wikipedia", "youtube"), start_date = "2016-01", end_date = "2019-12" ) add_object_keyword( keyword = c("apple", "facebook", "google", "microsoft"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = c("gmail", "maps", "news", "translate", "weather", "wikipedia", "youtube"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = c("amazon", "apple", "facebook", "google", "microsoft", "netflix", "twitter"), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = list( c("gmail", "maps", "news"), c("translate", "weather", "wikipedia", "youtube") ), start_date = "2016-01", end_date = "2019-12" ) add_control_keyword( keyword = list( c("amazon", "apple", "facebook", "google"), c("microsoft", "netflix", "twitter") ), start_date = "2016-01", end_date = "2019-12" ) # search topics add_control_keyword( keyword = c("%2Fm%2F02q_bk", "%2Fm%2F055t58", "%2Fm%2F025sndk", "%2Fm%2F0d07ph", "%2Fm%2F09jcvs"), start_date = "2016-01", end_date = "2019-12" ) # This adds the following topics: Gmail, Google Maps, Google Translate, Wikipedia, YouTube ## End(Not run)
The function adds a new set of locations for downloads and computations to the database. The location set serves as input for all download and computation functions.
add_locations(locations, type, export = TRUE)add_locations(locations, type, export = TRUE)
locations |
Locations that should be added as set of locations. Vector of
type |
type |
Name of the location set that should be added. Object of type
|
export |
Indicator whether the new location set should be directly
exported to the package environment |
Location sets control the locations for which data is downloaded or to which computations are applied. By adding new location sets, the default sets countries and us_states can be expanded by additional sets. Thereby, users can compute DOI within a region (e.g., adding EU countries as a set) or single countries (e.g., adding regions of France as a set). Download and computation functions check whether data for a location already exists. Therefore, data will not be duplicated when location data already exists from another set.
Message that the location set has been created successfully. Location data is written to table data_locations.
Unfortunately, the Google Trends API cannot handle the location "NA - Namibia". Therefore, the location will be dropped automatically.
## Not run: add_locations(locations = c("AT", "CH", "DE"), type = "DACH") ## End(Not run)## Not run: add_locations(locations = c("AT", "CH", "DE"), type = "DACH") ## End(Not run)
The function allows to add synonyms for object keywords. Sometimes, objects
of interest can be searched with different keywords on Google e.g., FC Bayern
for Bayern Munich. Search scores for keywords that are added as synonyms are
aggregated when running compute_score. The function allows to add
synonyms for a single keyword at a time.
add_synonym(keyword, synonym) ## S3 method for class 'character' add_synonym(keyword, synonym) ## S3 method for class 'list' add_synonym(keyword, synonym)add_synonym(keyword, synonym) ## S3 method for class 'character' add_synonym(keyword, synonym) ## S3 method for class 'list' add_synonym(keyword, synonym)
keyword |
Keyword of type |
synonym |
Synonym of type |
Message that the synonym has been added successfully. Synonym data is written to table keyword_synonyms.
To avoid trailing spaces stringr::str_squish is automatically
applied to all keywords and synonyms.
## Not run: add_synonym( keyword = "fc bayern", synonym = "bayern munich" ) ## End(Not run)## Not run: add_synonym( keyword = "fc bayern", synonym = "bayern munich" ) ## End(Not run)
The function aggregates search scores for object keywords across synonyms
defined through add_synonyms. Users should run this funciton after
running compute_score. Then, search scores will be added and the synonym
terms removed from the data_score table. For efficiency reasons, the
function uses purrr::map to iterate across batch_o.
aggregate_synonyms(control, vacuum = TRUE)aggregate_synonyms(control, vacuum = TRUE)
control |
Control batch for which the data is downloaded. Object
of type |
vacuum |
Indicator whether the function should call |
The function proceeds in five steps:
Export data from the database using purrr::map_dfr
Aggreagate score data from synonyms to main term
Remove batches that include synonyms from the database using purrr::walk
If vacuum = TRUE, run vacuum_data() on the database
Write aggregated data to the database using purrr::walk
During these steps, map_dfr and walk calls return progress information
through .progress = TRUE.
Message that the aggregation of synonyms is complete. During the
aggregation process, messages regarding each step. The function shows the
purrr::map_dfr and purrr:walk progress bars.
The aggregation of synonyms can be very time-consuming when using a large
dataset! Particulalry, the vacuum_data() call can require substantial time
to run.
## Not run: compute_score(object = 1:2, control = 1) aggregate_synonyms(control = 1, vacuum = FALSE) ## End(Not run)## Not run: compute_score(object = 1:2, control = 1) aggregate_synonyms(control = 1, vacuum = FALSE) ## End(Not run)
The table batch_keywords contains the keywords for each batch. Each
line contains one keyword, the type of the batch (i.e., control
or object) and the id of the batch to which the keyword is assigned.
Keywords can be added with the function add_keywords. The function
start_db exports the table batch_keywords as objects
keywords_control and keywords_object to the package environment
gt.env.
Example data for the table batch_keywords is available as R object
example_keywords.
example_keywordsexample_keywords
A tibble with 19 rows and 3 variables:
Column of type character showing the type of each batch,
takes "control" for control batches and "object" for object batches.
Column of type integer showing the number of each
batch.
Column of type character showing the keywords
included in each batch.
The table batch_time contains the time period for which data is
downloaded for each batch. Each line contains the start date, the
end date, the type of the batch (i.e., control or object), and the id of
the batch to which the time period is assigned. Start and end dates take
the form "YYYY-MM". Time periods are added automatically through
the function add_keywords. The function start_db exports the
table batch_time as objects time_control and time_object
to .GlobalEnv.
Example data for the table batch_time is available as R object
example_time.
example_timeexample_time
A tibble with 5 rows and 3 variables:
Column of type character showing the type of each batch,
takes "control" for control batches and "object" for object batches.
Column of type integer showing number of each batch.
Column of type character showing the start date for
each batch as "YYYY-MM".
Column of type character showing the end date for
each batch as "YYYY-MM".
The function computes degree of internationalization (DOI) for object keywords. Degree of internationalization is measured based on the distribution of country search scores.
compute_doi(object, control = 1, locations = "countries") ## S3 method for class 'numeric' compute_doi(object, control = 1, locations = "countries") ## S3 method for class 'list' compute_doi(object, control = 1, locations = "countries")compute_doi(object, control = 1, locations = "countries") ## S3 method for class 'numeric' compute_doi(object, control = 1, locations = "countries") ## S3 method for class 'list' compute_doi(object, control = 1, locations = "countries")
object |
Object batch for which the keyword-country data
is aggregated and DOI is computed. Object of type |
control |
Control batch for which the search score is used. Object
of type |
locations |
List of locations for which the search score is used.
Object of type |
The function uses an inverted Gini-coefficient as measure for the degree of internationalization. The more uniform the distribution of search scores across all countries, the higher the inverted Gini-coefficient and the greater the degree of internationalization. In addition to the Gini-coefficient, the package uses inverted Herfindahl index and inverted Entropy as measures for internationalization.
Message that data was aggregated successfully. Data is written to table data_doi.
## Not run: compute_doi( object = 1, control = 1, locations = "countries" ) compute_doi( object = as.list(1:5), control = 1, locations = "countries" ) ## End(Not run)## Not run: compute_doi( object = 1, control = 1, locations = "countries" ) compute_doi( object = as.list(1:5), control = 1, locations = "countries" ) ## End(Not run)
The function computes search scores for object keywords. Search volumes for
control and object batches are mapped to the same base. Next, search volumes
for object batches are divided by the sum of search volumes for the
respective control batch. compute_voi computes volume of
internationalization (VOI) as global search scores.
compute_score(object, control = 1, locations = gt.env$countries) ## S3 method for class 'numeric' compute_score(object, control = 1, locations = gt.env$countries) ## S3 method for class 'list' compute_score(object, control = 1, locations = gt.env$countries) compute_voi(object, control = 1)compute_score(object, control = 1, locations = gt.env$countries) ## S3 method for class 'numeric' compute_score(object, control = 1, locations = gt.env$countries) ## S3 method for class 'list' compute_score(object, control = 1, locations = gt.env$countries) compute_voi(object, control = 1)
object |
Object batch for which the data is downloaded. Object
of type |
control |
Control batch for which the data is downloaded. Object
of type |
locations |
List of countries or regions for which the data is
downloaded. Refers to lists generated in |
The search score computation proceeds in four steps. First, the function aggregates all search volumes to monthly data. Next, it follows the procedure outlined by Castelnuovo and Tran (2017, pp. A1-A2) to map control and object data. After the mapping, object search volumes are divided by the sum of control search volumes in the respective control batch. We use the sum of search volumes for a set of control keywords, rather than the search volumes for a single control keyword, to smooth-out variation in the underlying control data.
Castelnuovo, E. & Tran, T. D. 2017. Google It Up! A Google Trends-based Uncertainty index for the United States and Australia. Economics Letters, 161: 149-153.
Message that data has been computed successfully. Data is written to table data_score.
When synonyms were specified through add_synonym, run aggregate_synonyms
to add their search scores to the main term.
## Not run: compute_score( object = 1, control = 1, locations = countries ) compute_voi( object = 1, control = 1 ) compute_score( object = as.list(1:5), control = 1, locations = countries ) ## End(Not run)## Not run: compute_score( object = 1, control = 1, locations = countries ) compute_voi( object = 1, control = 1 ) compute_score( object = as.list(1:5), control = 1, locations = countries ) ## End(Not run)
A character vector that includes ISO2 codes for all countries with a share in global GDP >= 0.1\ World Development Indicators database. The data includes:
AE
AO
AR
AT
AU
BD
BE
BR
CA
CH
CL
CN
CO
CU
CZ
DE
DK
DO
DZ
EC
EG
ES
ET
FI
FR
GB
GR
HK
HU
ID
IE
IL
IN
IQ
IR
IT
JP
KR
KW
KZ
LK
MA
MX
MY
NG
NL
NO
NZ
OM
PE
PH
PK
PL
PR
PT
QA
RO
RU
SA
SD
SE
SG
SK
TH
TR
TW
UA
US
UZ
VN
ZA
countriescountries
An object of class character of length 71.
A data.frame that includes ISO2 codes and country names for all countries and locations in the World Bank's World Development Indicators database.
countries_wdicountries_wdi
An object of class data.frame with 299 rows and 2 columns.
The table data_control contains the downloaded data for each control
batch. Each line contains the search hits for each keyword in a
control batch for a given location and date. Global data
gets the value world as location. Data is downloaded and automatically
written to the table through the function download_control. The
function start_db exports the table data_control as database
connection tbl_control to the package environment gt.env. Users
can access the database table through dplyr::tbl.
The sample data included in data_control was simulated based on actual
Google Trends data.
Example data for the table data_control is available as R object
example_control.
example_controlexample_control
A tibble with 2,400 rows and 5 variables:
Column of type character showing the ISO2 code of
the country or region for which the data was downloaded.
Column of type character showing the keyword for
which the data was downloaded.
Column of type integer showing the date for which the
data was downloaded. Can be transformed into date format with
lubridate::as_date.\item{hits}{Column of typedoubleshowing search volumes for the respective location-keyword-date combination.} \item{batch}{Column of typeinteger' showing the number of each
batch.
https://trends.google.com/trends/
The table data_doi contains the degree of internationalization (DOI)
for each object batch. Each line contains the DOI computed as inverted
gini coefficient, as inverted hhi, or inverted entropy
for each keyword in an object batch_o for a given date.
The column batch_c indicates the control batch that has been used
as baseline for mapping. Column locations indicates which set of
locations was used to compute the distribution of search scores.
DOI is computed and automatically written to the table with the
function compute_doi. The function start_db exports the table
data_doi as database connection tbl_doi to the package environment
gt.env. Users can access the database table through dplyr::tbl.
The sample data included in data_doi was simulated based on actual
Google Trends data.
example_doiexample_doi
A tibble with 4,320 rows and 9 variables:
Column of type character showing the keyword for
which the data was computed.
Column of type integer showing the date for which the
data was computed Can be transformed into date format with
lubridate::as_date.
Column of type double showing the DOI computed as
inverted Gini coefficient of the search score distribution from
data_score.
Column of type double showing the DOI computed as
inverted Herfindahl-Hirschman index of the search score distribution from
data_score.
Column of type double showing the DOI computed as
inverted Entropy measure for the search score distribution from
data_score.
Column of type integer showing the number of each
control batch.
Column of type integer showing the number of each
object batch.
Column of type character showing the list of
locations for which the search score distribution is used.
The table data_object contains the downloaded data for each object
batch. Each line contains the search hits for each keyword in
an object batch_o for a given location and date. The
column batch_c indicates the control batch to which the data will be
mapped. Global data takes the value world as location. Data is
downloaded and automatically written to the table through the function
download_object. The function start_db exports the table
data_object as database connection tbl_object to the package
environment gt.env. Users can access the database table through
dplyr::tbl.
The sample data included in data_object was simulated based on actual
Google Trends data.
Example data for the table data_object is available as R object
example_object.
example_objectexample_object
A tibble with 8,640 rows and 6 variables:
Column of type character showing the ISO2 code of
the country or region for which the data was downloaded.
Column of type character showing the keyword for
which the data was downloaded.
Column of type integer showing the date for which the
data was downloaded. Can be transformed into date format with
lubridate::as_date.
Column of type double showing search volumes for the
respective location-keyword-date combination.
Column of type integer showing the number of each
control batch.
Column of type integer showing the number of each
object batch.
https://trends.google.com/trends/
The table data_score contains the search scores for each object batch.
Each line contains the search score (score) for each keyword in an
object batch_o for a given location and date. The column
batch_c indicates the control batch that has been used as baseline
for mapping. Global data takes the value world as location. Search
scores are computed and automatically written to the table with the function
compute_score. The function start_db exports the table
data_score as database connection tbl_score to
the package environment gt.env. Users can access the database
table through dplyr::tbl.
The sample data included in data_score was simulated based on actual
Google Trends data.
Example data for the table data_score is available as R object
example_score.
example_scoreexample_score
A tibble with 6,000 rows and 7 variables:
Column of type character showing the ISO2 code of
the country or region for which the data was computed.
Column of type character showing the keyword for
which the data was downloaded.
Column of type integer showing the date for which the
data was computed Can be transformed into date format with
lubridate::as_date.
Column of type double showing search score for the
respective location-keyword-date combination.
Column of type integer showing the number of each
control batch.
Column of type integer showing the number of each
object batch.
The function closes the connection to the database file db/globaltrends_db.sqlite in the working directory.
disconnect_db()disconnect_db()
Message that disconnection was successful.
SQLite databases only allow one writer at any instant in time. To run parallel downloads use one database for each download client and merge them once all downloads are complete.
## Not run: disconnect_db() ## End(Not run)## Not run: disconnect_db() ## End(Not run)
The function downloads search volumes from Google Trends for a control
batch in a set of locations. Data is automatically written to table
data_control. For download_control_global the input
location is automatically set to world.
download_control(control, locations = gt.env$countries) ## S3 method for class 'numeric' download_control(control, locations = gt.env$countries) ## S3 method for class 'list' download_control(control, locations = gt.env$countries) download_control_global(control)download_control(control, locations = gt.env$countries) ## S3 method for class 'numeric' download_control(control, locations = gt.env$countries) ## S3 method for class 'list' download_control(control, locations = gt.env$countries) download_control_global(control)
control |
Control batch for which the data is downloaded. Object
of type |
locations |
List of countries or regions for which the data is
downloaded. Refers to lists generated in |
Downloads through the Google Trends API are made through
gtrendsR::gtrends. Each control batch can consist of up to five
keywords and is predefined in tables batch_keywords and
batch_time through add_keywords. The download for a single
keyword batch for a single location takes about 30 seconds. This includes a
randomized waiting period of 5-10 seconds between downloads. Depending on
the frequency of downloads, Google Trends might block users for some time
(about 1,500 downloads within 24 hours). In this case, download_control
waits 60 minutes before it retries the download.
Message that data has been downloaded successfully. Data is written to table data_control.
We advise against the usage of category codes in downloads. If you use categories to narrow the context of keyword usage, these categories are applied to ALL keywords in the batch. This applies to control keywords as well as object keywords and can result in unintended behavior.
## Not run: download_control( control = 1, locations = countries ) download_control( control = as.list(1:5), locations = countries ) ## End(Not run)## Not run: download_control( control = 1, locations = countries ) download_control( control = as.list(1:5), locations = countries ) ## End(Not run)
The function downloads search volumes from Google Trends for an object batch
(batch_o) and one keyword from a control batch (batch_c) in a
set of locations. Data is automatically written to table
data_object. For download_object_global the input
location is automatically set to world.
download_object(object, control = 1, locations = gt.env$countries) ## S3 method for class 'numeric' download_object(object, control = 1, locations = gt.env$countries) ## S3 method for class 'list' download_object(object, control = 1, locations = gt.env$countries) download_object_global(object, control = 1)download_object(object, control = 1, locations = gt.env$countries) ## S3 method for class 'numeric' download_object(object, control = 1, locations = gt.env$countries) ## S3 method for class 'list' download_object(object, control = 1, locations = gt.env$countries) download_object_global(object, control = 1)
object |
Object batch for which the data is downloaded. Object
of type |
control |
Control batch that is used for mapping. Object of type
|
locations |
List of countries or regions for which the data is
downloaded. Refers to lists generated in |
Downloads through the Google Trends API are made through
gtrendsR::gtrends. Each object batch can consist of up to four
keywords and is predefined in tables batch_keywords and
batch_time through add_keywords. In addition, one control
keyword is added to each object batch. The control keyword then allows a
mapping between search volumes for control keywords stored in
data_control and search volumes for object keywords. The download for
a single keyword batch for a single location takes about 30 seconds. This
includes a randomized waiting period of 5-10 seconds between downloads.
Depending on the frequency of downloads, Google Trends might block users
for some time (about 1,500 downloads within 24 hours). In this case,
download_object waits 60 minutes before it retries the download.
Message that data was downloaded successfully. Data is written to table data_object.
We advise against the usage of category codes in downloads. If you use categories to narrow the context of keyword usage, these categories are applied to ALL keywords in the batch. This applies to control keywords as well as object keywords and can result in unintended behavior.
## Not run: download_object( object = 1, locations = countries ) download_object( object = as.list(1:5), locations = countries ) ## End(Not run)## Not run: download_object( object = 1, locations = countries ) download_object( object = as.list(1:5), locations = countries ) ## End(Not run)
The function allows to export data from database tables. In combination with various write functions in R, the functions allow exports from the database to local files.
export_control(control = NULL, location = NULL) export_control_global(control = NULL) export_object(keyword = NULL, object = NULL, control = NULL, location = NULL) export_object_global(keyword = NULL, object = NULL, control = NULL) export_score(keyword = NULL, object = NULL, control = NULL, location = NULL) export_voi(keyword = NULL, object = NULL, control = NULL) export_doi(keyword = NULL, object = NULL, control = NULL, locations = NULL)export_control(control = NULL, location = NULL) export_control_global(control = NULL) export_object(keyword = NULL, object = NULL, control = NULL, location = NULL) export_object_global(keyword = NULL, object = NULL, control = NULL) export_score(keyword = NULL, object = NULL, control = NULL, location = NULL) export_voi(keyword = NULL, object = NULL, control = NULL) export_doi(keyword = NULL, object = NULL, control = NULL, locations = NULL)
control |
Control batch number for which data should be exported. Only
for |
location |
List of locations for which the data is exported. Refers to
lists generated in |
keyword |
Object keywords for which data should be exported. Object or
list of objects of type |
object |
Object batch number for which data should be exported. |
locations |
List of locations for which the data is exported. Refers to
names of lists generated in |
Exports can be filtered by keyword, object, control, location,
locations, or type. Not all filters are applicable for all
functions. When filter keyword and object are used together,
keyword overrules object. When supplying NULL as input, no filter is
applied to the variable.
The functions export and filter the respective database tables.
export_control and export_control_global export data from table
data_control with columns location, keyword, date, hits, control. Object
of class "data.frame". Methods are applied based on input control.
export_object and export_object_global export data from table
data_object with columns location, keyword, date, hits, object. Object of
class "data.frame". Methods are applied based on input keyword.
export_score exports data from table data_score with
columns location, keyword, date, score, control,
object. Object of class "data.frame". Methods are applied based on input
keyword.
export_voi exports data from table data_score with
columns keyword, date, score, control, filters for
location == "world". Object of class "data.frame". Methods are applied
based on input keyword.
export_doi exports data from table data_doi with columns
keyword, date, gini, hhi, entropy, control, object, locations. Object
of class "data.frame". Methods are applied based on input keyword.
## Not run: export_control(control = 2) export_object( keyword = "manchester united", locations = countries ) export_object( keyword = c("manchester united", "real madrid") ) export_object( keyword = list("manchester united", "real madrid") ) export_score( object = 3, control = 1, location = us_states ) %>% readr::write_csv("data_score.csv") export_doi( keyword = "manchester united", control = 2, locations = "us_states" ) %>% writexl::write_xlsx("data_doi.xlsx") ## End(Not run)## Not run: export_control(control = 2) export_object( keyword = "manchester united", locations = countries ) export_object( keyword = c("manchester united", "real madrid") ) export_object( keyword = list("manchester united", "real madrid") ) export_score( object = 3, control = 1, location = us_states ) %>% readr::write_csv("data_score.csv") export_doi( keyword = "manchester united", control = 2, locations = "us_states" ) %>% writexl::write_xlsx("data_doi.xlsx") ## End(Not run)
The environment gt.env contains all package-related data objects, such as
the handle for the SQLite database file or connections to tables. The object
contains:
globaltrends_db: Handle for the SQLite database file.
tbl_locations: Connection to table that contains the lists of locations saved in the database.
tbl_keywords: Connection to table that contains the lists of keywords saved in the database.
tbl_time: Connection to table that contains the lists of batch times saved in the database.
tbl_synonyms: Connection to table that contains the lists of keyword synonyms saved in the database.
tbl_doi: Connection to table that contains the DOI data saved in the database.
tbl_control: Connection to table that contains data on search volume for control terms saved in the database.
tbl_object: Connection to table that contains data on search volume for object terms saved in the database.
tbl_score: Connection to table that contains data on search scores saved in the database.
keywords_control: Tibble that contains all keywords per control batch.
time_control: Tibble that contains all batch times per control batch.
keywords_object: Tibble that contains all keywords per object batch.
time_object: Tibble that contains all batch times per object batch.
keyword_synonyms: Tibble that contains all keyword/synonym combinations.
query_wait: Number of seconds to wait between queries (default = 0.1s).
py_setup: TRUE/FALSE indicator whether initialize_python has been called.
gt.envgt.env
An object of class environment of length 16.
The function creates a new database for the globaltrends package and
creates all necessary tables within the database.
initialize_db()initialize_db()
The function creates a new SQLite database for the globaltrends
package. The database is saved as file db/globaltrends_db.sqlite in
the working directory. If the folder db does not exists in the working
directory, the folder is created. If the database already exists in the
working directory, the function exits with an error. Within the database all
tables are created and the default location sets are added to the respective
table:
countries - all countries with a share in global GDP >= 0.1\ in 2018.
us_states - all US federal states and Washington DC.
After creating the database, the function disconnects from the database.
Database is created.
SQLite databases only allow one writer at any instant in time. To run parallel downloads use one database for each download client and merge them once all downloads are complete.
## Not run: initialize_db() ## End(Not run)## Not run: initialize_db() ## End(Not run)
The function initializes the Python session required to download data from Google Trends using the Google Trends research API. The user can define an API key and either a Conda or Python environment.
initialize_python(api_key, conda_env = NULL, python_env = NULL)initialize_python(api_key, conda_env = NULL, python_env = NULL)
api_key |
API key obtained from Google. Object of type |
conda_env |
Storage location of a Conda environment. Object of type
|
python_env |
Storage location of a Python environment. Object of type
|
To use this API acces, you must:
Apply for API access online. and generate an API key in your Google developer console.
Create a Python or Conda environment where you install the
google-api-python-client package with pip.
The functions sets the environment variable gt.env to TRUE. This channels
downloads to the API. Otherwise, globaltrends uses gtrendsR::gtrends for
downloads.
## Not run: initialize_python( api_key = "XXX", conda_env = "/xxx/venv" ) ## End(Not run)## Not run: initialize_python( api_key = "XXX", conda_env = "/xxx/venv" ) ## End(Not run)
The function removes data from database tables for control or object batches.
remove_data(table, control = NULL, object = NULL) vacuum_data()remove_data(table, control = NULL, object = NULL) vacuum_data()
table |
Database table from which the batch should be removed. Object
of type |
control |
Control batch for which the data is removed Object
of type |
object |
Object batch for which the data is removed Object
of type |
The function removes data "greedily": all data that builds on the deleted data is removed. For example, when data from data_control is removed data in data_object that maps to this control batch is also removed. The dependency structure works as follows: batch_keyword / batch_time -> data_control -> data_object -> data_score -> data_doi.
After using remove_data, run vacuum_data to free-up unused memory in
the database file. Depending on the database size, vacuum_data might
take some minutes for execution.
Message that data has been removed successfully. Data is removed from database tables.
## Not run: remove_data( table = "batch_keywords", control = 1 ) remove_data( table = "data_score", control = 1, object = 1 ) vacuum_data() ## End(Not run)## Not run: remove_data( table = "batch_keywords", control = 1 ) remove_data( table = "data_score", control = 1, object = 1 ) vacuum_data() ## End(Not run)
The function connects to the database file db/globaltrends_db.sqlite
in the working directory. After connecting to the database connections to the
database tables (through dplyr::tbl) are created. Data from the tables
batch_keywords and batch_time are exported to the tibble
objects keywords_control, keywords_object, time_control,
and time_object.
start_db()start_db()
The function exports the following objects to the package environment globaltrends_db:
globaltrends_db A DBIConnection object, as returned by
DBI::dbConnect(), connecting to the SQLite database in the working
directory
tbl_doi A remote data source pointing to the table data_doi in the connected SQLite database
tbl_control A remote data source pointing to the table data_control in the connected SQLite database
tbl_mapping A remote data source pointing to the table data_mapping in the connected SQLite database
tbl_object A remote data source pointing to the table data_object in the connected SQLite database
tbl_score A remote data source pointing to the table data_score in the connected SQLite database
countries A character vector containing ISO2 country codes of
countries that add at least 0.1\
us_states A character vector containing ISO2 regional codes of
US states
keywords_control A tibble containing keywords of control
batches
time_control A tibble containing times of control batches
keywords_object A tibble containing keywords of object batches
time_object A tibble containing times of control batches
keyword_synonyms A tibble containing synonymous keywords
SQLite databases only allow one writer at any instant in time. To run parallel downloads use one database for each download client and merge them once all downloads are complete.
## Not run: start_db() ## End(Not run)## Not run: start_db() ## End(Not run)
A character vector that includes ISO2 codes for all US federal states and Washington DC. The data includes:
US-AL
US-AK
US-AZ
US-AR
US-CA
US-CO
US-CT
US-DE
US-FL
US-GA
US-HI
US-ID
US-IL
US-IN
US-IA
US-KS
US-KY
US-LA
US-ME
US-MD
US-MA
US-MI
US-MN
US-MS
US-MO
US-MT
US-NE
US-NV
US-NH
US-NJ
US-NM
US-NY
US-NC
US-ND
US-OH
US-OK
US-OR
US-PA
US-RI
US-SC
US-SD
US-TN
US-TX
US-UT
US-VT
US-VA
US-WA
US-WV
US-WI
US-WY
US-DC
us_statesus_states
An object of class character of length 51.