NEWS
globaltrends 0.2.1 (2026-07-08)
Data Structure
- Change from sqlite/parquet files to
data.table for performance reasons
- All
globaltrends data is stored in a single, non-compressed RDS file
- The data is stored in a list objectiv with the following
data.table objects:
batch_keywords
batch_time
data_control
data_object
data_score
data_doi
data_locations
data_region
data_related
keyword_synonyms
- All
globaltrends data is loaded into memory as a list of data.table objects
- This makes the storage structure incompatiable with the structure used up to
0.2.0
- It is possible to integrate the data manually. To this end, extract the data from the parquet files, transform it to
data.table objects and enter it into the respective elements in the gt.env environment.
- Drop
vacuum_data() function, since the data architecture is not built on a database file any more
Efficiency
compute_score() now persists in-memory data to disk after every 1,000 computed locations, matching the periodic persistence already used by the download_xxx() functions
Testing
- All tests now run fully isolated, in their own temporary directory, with automatic cleanup (
withr) - no shared state or leftover files between tests
- Live Google Trends API calls in the test suite are now opt-in (set
GLOBALTRENDS_LIVE_TESTS=1), so automated test runs are offline and deterministic by default
- Added test coverage for previously untested internals: API retry/backoff logic, quota handling, input validators, database persistence, and
initialize_python()
- Enabled parallel test execution (
Config/testthat/parallel: true)
globaltrends 0.2.0
- General revision and improvement of code base
- General revision and improvement of tests
- General revision and improvement of documentation
- Inclusion of API-usage counter for downloads through Google Trends API (10,000 daily downloads)
Change in database handling
- Switch from sqlite to parquet
- Database tables are stored as individual parquet files
start_db() loads the parquet files to memory
- All database actions are conducted in memory and later written to parquet
- The download functions write in-memory data to parquet after every 1,000 downloads
Technical issues and bug fixes
- Updated dependency to testthat 3.0.0
- Improved error handling for Google Trends API calls:
.get_trend() now catches three classes of errors from the Research API backend (HTTP via reticulate) instead of crashing with a Python traceback
- Quota exceeded (HTTP 429): writes in-memory data to parquet and then aborts with a clear message asking the user to wait for the quota to reset.
- Invalid argument (HTTP 400): skips the offending term/location/date combination silently and continues.
- Timeout (WinError 10060): skips the affected location and continues; the missing data can be filled by re-running download_object().
globaltrends 0.1.0
Change in downloading approach
- Run downloads from Google Trends through the official Google Trends API
- Users can choose to download through the official API or the
gtrendsrpackage
- If users call
initialized_python(), the downloads will be made through
the official Google Trends API
- If users run downloads without calling
initialized_python(), the downloads
will be made through the gtrendsrpackage
- Control the waiting time between queries through the environmental variable
gt.env$query_wait (default value = 0.1s)
Package simplifications
- Remove time-series adjustments from
compute_score for package simplification
- Remove
get_abnorm_hist function for package simplification
- Remove plotting functions and export classes for package simplification
- Accelerate
compute_score
- Accelerate handling of synonyms in separate function
aggregate_synonyms
Technical issues and bug fixes
- Increase dependencies to
dplyr 1.1.1 and replace the multiple argument by
relationship
- Fixed issues in sample data
globaltrends 0.0.14 (2023-03-06)
- Remove dependency on
glue package, calls of glue::glue are replaced with paste and paste0
- Remove dependency on
ineq package, calls of ineq::ineq are replaced with internal functions
- Use functions from
DBI 1.1.0: dbCreateTable and dbAppendTable
- Remove the function
plot_map from the package:
- The function proofed too inflexible, only world map was included
- We point users to the functions from the packages
maps and ggmap
- Remove dependency on the
maps package
- Remove reference to non-existent table
data_global in function remove_data
globaltrends 0.0.13 (2023-02-23)
- Set minimum dependency of
stats and utils to version 3.5.0 in line with minimum R dependency
- Store globaltrends logo in the vignettes folder
- Add explanation of Google's data preparation methodology to vignette
- Change wait intervals for status responses != 200 and add message about automatic retry of download
- Status == 500: wait 1 second
- All other responses: wait 60 seconds
- Add function
vacuum_data to free unused memory after remove_data
- Remove all usage of
.data to comply with tidyselect 1.2.0, this applies to calls of:
dplyr::rename
dplyr::select
purrr::map
purrr::walk
tidyr::nest
tidyr::pivot_longer
tidyr::pivot_wider
tidyr::unnest
- As a consequence of the changes in
tidyselect 1.2.0, several objects are defined as global variables (see globals.r for details)
- Replace
size with linewidth to comply with ggplot2 3.4.0
- Add the argument
multiple = "all" and multiple = "error" to comply with dplyr 1.1.0
- Facilitate handling of location = "NA - Namibia", the function
add_locations does automatically drop the location
- Remove dependency on
WDI package, references to WDI::WDI_data are replaced by data objects countries and countries_wdi
globaltrends 0.0.12 (2022-06-23)
- Stop direct exports from functions to .GlobalEnv
- Exports of objects (e.g., SQL connections, keyword tables) are redirected to package environment
gt.env
- Add handle for Namibia ISO code
globaltrends 0.0.11
- Add function plot_map()
- Add automatic stringr::str_squish for keywords and synonyms
- Add documentation on usage of search topics
globaltrends 0.0.10
- Change indication and handling of synonyms (synonyms are now ignored in
export_score and export_voi)
globaltrends 0.0.9
- Fasten computation of search scores for data with synonyms
- Fix bugs for control keywords that have a mean of 0 for their search volume time series
globaltrends 0.0.8
- Adapt
export_xxx for vector and list inputs
- Optimize RAM usage of
export_xxx
- Check whether "db/globaltrends_db.sqlite" file exists in working directory
- Inclusion of "workaround" plot functions that set class
xxx automatically:
plot_xxx_box
plot_xxx_bar
plot_xxx_ts
globaltrends 0.0.7
- Change waiting times for errors
- Status Code != 200/Limit exceeded -> 60 seconds wait
- Status Code == 500 -> 1 second wait
globaltrends 0.0.6
- To distinguish them from the actual database tables, names of the example data
object were adapted. Documentation is still available for the respective database
table.
batch_keywords -> example_keywords
batch_time -> example_time
data_control -> example_control
data_doi -> example_doi
data_object -> example_object
data_score -> example_score
- Waiting period between downloads reduced from 20-30 seconds to 5-10 seconds
globaltrends 0.0.5
- Added classes for output from
export_xxx
export_score -> class("exp_score")
export_voi -> class("exp_voi")
export_doi -> class("exp_doi")
export_xxx + get_abnorm_hist supersedes export_xxx_change
- method for
export_score -> class("exp_score")
- method for
export_voi -> class("exp_voi")
- method for
export_doi -> class("exp_doi")
plot_bar supersedes plot_score
- method for
export_score -> class("exp_score")
- method for
export_score -> get_abnorm_hist -> class("abnorm_score")
plot_ts supersedes plot_voi_ts and plot_doi_ts
- method for
export_score -> class("exp_score")
- method for
export_score -> get_abnorm_hist -> class("abnorm_score")
- method for
export_voi -> class("exp_voi")
- method for
export_voi -> get_abnorm_hist -> class("abnorm_voi")
- method for
export_doi -> class("exp_doi")
- method for
export_doi -> get_abnorm_hist -> class("abnorm_doi")
plot_box supersedes plot_voi_box and plot_doi_box
- method for
export_score -> class("exp_score")
- method for
export_score -> get_abnorm_hist -> class("abnorm_score")
- method for
export_voi -> class("exp_voi")
- method for
export_voi -> get_abnorm_hist -> class("abnorm_voi")
- method for
export_doi -> class("exp_doi")
- method for
export_doi -> get_abnorm_hist -> class("abnorm_doi")