Frames

Untitled

0
1# Change Log
2All notable changes are kept in this file. All changes made should be added to the section called
3`Unreleased`. Once a new release is made this file will be updated to create a new `Unreleased`
4section for the next release.
5
6For more information about this file see also [Keep a Changelog](http://keepachangelog.com/) .
7
8## [Unreleased]
9
10### Fixes
11- Fixed issue that prevented modellauncher from working properly #2262
12- Use explicit namespacing (`package::function`) throughout `PEcAn.meta.analysis`. Otherwise, many of these functions would fail when trying to run a meta-analysis outside of the PEcAn workflow (i.e. without having loaded the packages first) (#2351).
13- Standardize how `PEcAn.DB` tests create database connections, and make sure tests work with both the newer `Postgres` and older `PostgreSQL` drivers (#2351).
14- Meta-analysis = "AUTO" now correctly skips the meta analysis if the PFT definition has not changed (#1217).
15- Replace deprecated `rlang::UQ` syntax with the recommended `!!`
16- Explicitly use `PEcAn.uncertainty::read.ensemble.output` in `PEcAn.utils::get.results`. Otherwise, it would sometimes use the deprecated `PEcAn.utils::read.ensemble.output` version.
17- History page would not pass the hostname parameter when showing a running workflow, this would result in the running page showing an error.
18- The `parse` option to `PEcAn.utils::read_web_config` had no effect when `expand` was TRUE (#2421).
19
20### Changed
21- Updated modules/rtm PROSPECT docs
22- Updated models/sipnet/R/model2netcdf.SIPNET.R to address issues in PR #2254
23- Improved testing (#2281). Automatic Travis CI builds of PEcAn on are now run using three versions of R in parallel. This should mean fewer issues with new releases and better backwards compatibility, but note that we still only guarantee full compatibility with the current release version of R. The tested versions are:
24 - `release`, the current public release of R (currently R 3.5). Build failures in this version are fixed before merging the change that caused them. When we say PEcAn is fully tested and working, this is the build we mean.
25 - `devel`, the newest available development build of R. We will fix issues with this version before the next major R release.
26 - `oldrel`, the previous major release of R (currently R 3.4). We will fix issues with this version as time allows, but we do not guarantee that it will stay compatible.
27- Reverting back from PR (#2137) to fix issues with MAAT wrappers.
28- Moved docker files for models into model specific folder, for example Dockerfile for sipnet now is in models/sipnet/Dockerfile.
29- `PEcAn.utils`:
30 - Remove, or make "Suggests", a bunch of relatively underutilized R package dependencies.
31- Add template for documentation issues and add button to edit book.
32- Conditionally skip unit tests for downloading raw met data or querying the database when the required remote connection is not available.
33- Reorganization of docker folder
34 - All dockerfiles now live in their own folder
35 - `scripts/generate_dependencies.R` is now used to generate dependencies for make and docker
36- In `PEcAn.DB::get.trait.data`, if `trait.names` is `NULL` or missing, use the traits for which at least one prior is available among the input list of PFTs. (Previously, we were getting this from the `PEcAn.utils::trait.dictionary`, which we are trying to deprecate #1747). (#2351)
37- Cleanup and improve logging and code readability in parts of `PEcAn.DB` related to getting trait data, including replacing many manual database queries with `dplyr` calls.
38- Reorganization of PEcAn documentation in accordance with isue #2253.
39- SIPNET now is installed from the source code managed in git
40
41### Added
42- Meta analysis functionality to not use greenhouse data.
43- Dockerize the BioCro model.
44- Added PRO4SAIL-D model, using existing 4SAIL src and coupling with PROSPECT-D Fortran code
45- Models will not advertise themselvs, so no need to register them a-priori with the database #2158
46- Added simple Docker container to show all containers that are available (http://localhost:8000/monitor/). This will also take care of registering the models with the BETY database.
47- Added unit tests for `met2model.<MODEL>` functions for most models.
48- Added MAESPA model to docker build
49- PEcAn has more robust support for `RPostgres::Postgres` backend. The backend is officially supported by `db.query`, and basic workflows run top-to-bottom with the `Postgres` backend. However, `RPostgreSQL` is still the default until we do more robust testing of all modules.
50- `PEcAn.DB::db.query` now optionally supports prepared statements (#395).
51- New function `PEcAn.DB::query_priors` that expands the functionality of `query.priors` by (1) accepting PFTs by name or ID and (2) allowing the user to request all possible combinations of the input PFTs and traits (i.e. `expand.grid(pfts, traits)`) or just the pairwise combinations (i.e. `pft[1]-trait[1], pft[2]-trait[2]`). This function also comes with more robust error handling and a set of unit tests (#2351).
52- New function `PEcAn.DB::query_pfts` for finding PFT IDs and types from the PFT name and (optionally) model type (#2351).
53- Run Travis integration tests with both Postgres and PostgreSQL drivers (#2351).
54- New function `PEcAn.utils::load_local` reads `Rdata` files into a named list (instead of into the current environment).
55
56### Removed
57- Removed unused function `PEcAn.visualization::points2county`, thus removing many indirect dependencies by no longer importing the `earth` package.
58- Removed package `PEcAn.data.mining` from the Make build. It can still be installed directly from R if desired, but is skipped by default because it is in early development, does not yet export any functions, and creates a dependency on the (large, often annoying to install) ImageMagick library.
59- Fully deprecate support for `MySQL` database driver. Now, only `PostgreSQL` (and, experimentally, `RPostgres`) are supported. With this, remove `RMySQL` dependency in several places.
60
61### Fixed
62- Replace deprecated `rlang::UQ` syntax with the recommended `!!`
63- Explicitly use `PEcAn.uncertainty::read.ensemble.output` in `PEcAn.utils::get.results`. Otherwise, it would sometimes use the deprecated `PEcAn.utils::read.ensemble.output` version.
64- `PEcAn.ED2::met2model.ED2` now skips processing of years for which all output files are already present (unless `overwrite = TRUE`). This prevents a lot of unnecessary work when extending an existing ED met record.
65
66## [1.7.0] - 2018-12-09
67
68### Fixes
69- Fixed minor bug in query.trait.data related to stem respiration covariates (https://github.com/PecanProject/pecan/issues/2269)
70- Removed google maps and replaced with leaflet #2105
71- Added ability to add a new site from web interface
72- Small updated to models/ed/R/model2netcdf.ED2.R to fix issue realted to writing the time_bounds time attribute. Needed to add a check for which file types exitst (e.g. -E-, -T-, etc) and only write the appropriate attribute(s).
73- Fixed error in `read_web_config` which would filter out all variables.
74- Docker:
75 - Make sure web interface posts RabbitMQ messages even after editing files (fixes #2151)
76 - Can specify name of docker cluster using PECAN_FQDN and PECAN_NAME (fixes #2128)
77 - Fixed issue where setting username/password for rabbitmq would break web submit (fixes #2185)
78 - data image only registers sipnet and ed, has all data pre-downloaded
79- ED2:
80 - Fix processing of `ed2in_tags` from XML. Now numbers (e.g. `<TRAIT_PLASTICITY_SCHEME>0</TRAIT_PLASTICITY_SCHEME>`) and numeric vectors (e.g. `<INCLUDE_THESE_PFT>9,10,11,12</INCLUDE_THESE_PFT>`) are correctly written to ED2IN _without_ quotes.
81
82### Added
83- NEW FEATURE: PEcAn R API (PR #2192). Features include:
84 - Modified `docker/receiver.py` to accept a `pecan_json` object containing a JSON version of the PEcAn settings. Can now Use RabbitMQ HTTP API (called from R with `httr`) to send a settings list (function `submit_workflow`)
85 - Helper functions to make it easier to build the settings object, and to register a new workflow.
86 - Helper functions for tracking workflow status
87 - Helper functions for accessing workflow output files through THREDDS. All files are accessible through `fileServer` (basically, direct download), and NetCDF files are also readable through OpenDAP.
88 - THREDDS catalog filter has been removed, so that _all_ workflow outputs are available to THREDDS.
89 - Added another `datasetScan` to the THREDDS catalog to search for `dbfiles`. Now, model inputs (e.g. meteorology files) are accessible via THREDDS as well.
90- Lots of new documentation for running PEcAn using Docker
91- Added Docker container with documentation #2160
92- Download method (`method`) argument for `data.atmosphere::download.CRUNCEP`, which defaults to `opendap` (as it was), but can be switched to the slower but more robust NetCDF subset (`ncss`).
93- In `download.CRUNCEP`, check target coordinate against the land-sea mask. If sea, pick the nearest land pixel within 1 degree of target. This facilitates doing runs at coastal sites that may get masked out.
94- Added a prototype of the THREDDS data server (TDS) to the PEcAn Docker stack.
95- Added portainer to the PEcAn Docker stack to easily look at running containers.
96- Added ability to specify short name for a host (hostlist->displayname)
97- Added `PEcAn.logger::print2string` function -- capture the output
98- Cleanup and enhancements to `PEcAn.utils::read.output`:
99 - Pass `variables = NULL` to try to read _all_ variables from file
100 - New argument `ncfiles` for passing file names explicitly (useful for remote file access where `list.files` doesn't work; e.g. THREDDS)
101 - Variable summary stats are only calculated if new argument `print_summary` is `TRUE` (default). Summary is rendered nicely as a variable x statistic matrix.
102 - New argument `verbose` (default = `FALSE`) to print out (`logger.debug`) at every variable and year
103 - Minor code cleanup for style (spacing, long lines, etc.) and logic (replace several `else` statements with early returns)
104- ED2:
105 - Add ability to pass arbitrary arguments to the ED binary through the `pecan.xml` (#2183; fixes #2146).
106 - Add new `model` tag `<all_pfts>`. If "false" (default), set ED2IN's `INCLUDE_THESE_PFT` to only PFTs explicitly configured through PEcAn. If "true", use all 17 of ED2's PFTs.
107 - Add new `model` tag `<barebones_ed2in>`. If "true", only write ED2IN tags, and do not include comment annotations. If "false" (default), try to transfer comments from ED2IN template to target ED2IN. Because of comments are written by matching line numbers, leaving this as "false" can lead to unexpected results whenever `<ed2in_tags>` contains tags missing from the `ED2IN` template.
108 - Add some additional documentation for ED2 `pecan.xml` tags.
109
110### Removed
111
112### Changed
113- Updated MAAT model model2netcdf.MAAT.R to reflect general changes to the netCDF time variable in PEcAn standard output. Added time_bounds attribute and variable. Updated inst/ scripts for created MAAT drivers from NGEE-Tropics met sources (WIP)
114- `PEcAn.utils::do_conversions` has been moved to `PEcAn.workflow::do_conversions`.
115 `PEcAn.utils::do_conversions` still works for now with a warning, but is deprecated and will be removed in the future.
116- Docker:
117 - Change base image for R code from `r-base` to `rocker/tidyverse:3.5.1`. This (1) saves build time (because many R packages and system dependencies are pre-installed), and (2) enhances reproducibility (because of the strict versioning rules of the `rocker` packages)
118 - Re-factor web interface RabbitMQ create connections and post messages into their own PHP functions.
119
120## [1.6.0] - 2018-09-01
121
122### Fixes
123- Updated model2netcdf.SIPNET() to address issue #2094. Revised netCDF time to be from 0-364./365. (if leap) so time would be properly parsed by R and python (cf tools)
124- Fixed output time variable in models/ed/R/model2netcdf.ED2.R to provide correct fractional DOY
125- Running tests for PEcAn.settings package no longer leaves empty temp directories in test folder (#2075)
126- Fixed issue #2064 which sends one met path to write.sa.config.
127- `PEcAn.data.land::soil_params` now accepts any 2 out of 3 texture components as documented, and correctly converts percentages to proportion (#2043).
128- Added missing ncdf4 library calls in model2netcdf.JULES
129
130### Added
131- Added download.LandTrendr.AGB and extract.LandTrendr.AGB functions in modules/data.remote
132- Added new time_bounds variable in SIPNET output netCDF files to define the exact start time and end time for each model timestep.
133- Updated models/ed/R/model2netcdf.ED2.R to include new time_bounds variable
134- Added a first vignette to models/maat with the plan to add more examples
135- Added scaling to documentation
136
137### Removed
138- Removed unused PEcAn.utils::counter(), which existed to increment a global variable that is also unused.
139
140### Changed
141- Updated models/fates/R/model2netcdf.FATES.R to increase supported model outputs. Added longname to nc file variables
142- Updated models/dalec/R/model2netcdf.DALEC.R to add time_bounds variable
143- Updated models/maat/R/write.config.MAAT.R to improve flow, remove bugs, and to work with the release version of the MAAT model.
144- Minor update to modules/data.atmosphere/R/met2CF.csv.R to include recursive=TRUE for outfolder. Seemed to work better
145- Updated models/maat/R/met2model.MAAT.R to include additional output variables, fix a bug, and conduct overall cleanup. Updated docs
146- Updated models/maat/R/model2netcdf.MAAT.R to work with the release version of the MAAT model. Other small MAAT wrapper code cleanup
147- Small change to modules/data.atmosphere/R/download.NARR_site.R to set parallel=TRUE to match documentation and sub-function calls
148
149
150## [1.6.0] - Not yet
151
152### Fixes
153- Fixed issue #1939 which corrects output time vector for FATES output
154- Update to read.output to look for and read only PEcAn formatted .nc output based on the pecan standard filename format of YYYY.nc. Solves issues with models such as FATES and dvm-dos-tem where the original model output is also in .nc file format and was not ignored by read.output, causing errors with output parsing and plotting with Shiny. Removed deprecated function convert.outputs
155- PEcAn.data.atmosphere:
156 - download.Geostreams is pickier about formatting start/end datess, for fewer surprises in result timestamps
157 - Fixed swapped lat/lon in met2CF.Geostreams
158 - download.GFDL now records reference date in time units field, as required by the CF met standard
159 - Reduced download.GFDL network load by not preloading dimension data
160 - Fixed spurious `No geonamesUsername set` warning by updating geonames package to development version
161- ED:
162 - Change all history parameter files to have zero storage respiration
163- missing_port_bety
164- dataone_download.R:
165
166 - Added functionality that spoofs our user address to prevent authentication errors with downloading files via wget.
167- Could not specify the port for BETY in config.php. Can now use `db_bety_port` to specify port.
168
169 - Added functionality that spoofs our user address to prevent authentication errors with downloading files via wget.
170
171- Data_Ingest_App:
172 - use `updateSelectizeInput` to populate `selectizeInput` with choices from BETYdb. This instantly loads the inputfields where other methods take minutes to load.
173
174
175### Added
176- sda.enkf function inside the `PEcAn.assim.sequential` package was replaced with the refactored version, while the original sda function can be found in the same package with the name of sda.enkf.original.
177- PEcAn.undertainty gains one new function (input.ens.gen) and three functions moved from PEcAn.utils (see "Changed" below)
178- IC workflow now has functionality to generate ensembles.
179- You can now generate ensembles for parameters and met separatly and using different methods.
180- Soil process is now capable of reading in soil data from gSSURGO databse.
181- In modules/rtm new function foursail() to interface with the 4SAIL Fortran code. To enable the use of 4SAIL with any version of PROSPECT (i.e. 4, 5, 5b, D) and custom soil/background reflectance inputs
182- Shiny/Dependency explorer
183- Explore the interdependencies between pecan packages/functions.
184- From history you can now select an old run and show the curl command to re-execute this run. This only works with runs submitted through web interface right now.
185- Experimental support for docker (#1028)
186
187- dataone_download.R:
188 - Added progress messages to indicate that the function is working during longer downloads via PEcAn logger.
189 - Store path to downloaded data as newdir_D1 so that the download app can call this path.
190
191- shiny/Data-Ingest
192 - Download data from DataONE to a temporary directory, display the contents of that directory, and then move files from the temporary directory to dbfiles
193 - Upload files from local machines (via drag and drop on some browsers), display files, and then move these files to a directory in dbfiles.
194 - Spinner displays when class "shiny-busy" is invoked during the dataONE download process. In this way, users can be sure that the app is live and is processing their request.
195 - Users can now input the name of the destination directory that they wish to create within dbfiles.
196 - Updated Travis.yml to include librdf0-dev so that it can download redland, datapack, and dataone.
197 - Added Data-Ingest UI (inputs, dbfiles, and formats record UI and some basic server side functionality are online)
198 - Modularized input record, format record, and dbfiles record into shiny modules. This allows the app to be greatly simplified to two, single-page workflows. These functions can also be used "plug-and-play" style elsewhere in PEcAn shiny apps to load in data.
199 - Replaced modularized input, format and dbfiles records with static "Ingest Workflow" page. On this page, the user can select either importing from dataONE or Uploading from local files. If creating a new format is necessary, the user can click "Create New Format" and a dropdown menu will walk them through this process.
200 - Selected files now autofill name value in input record workflow
201 - Store inputs and formats in the global environment
202 - "Test BETY" button allows users create a record in BETY with `dbfile.input.insert`
203 - Added `input.format.vars` to query the BETYdb
204 - New File: `helper.R`
205 - New Function: `auto.name.directory` This function uses the format_name and the site_id for a given input to create a directory name in the style of other dbfiles names.
206 - `Next Step` buttons progress workflow programmatically
207 - New formats-variables UI allows user to create a table of formats-variable records before completing the ingest process
208 - Two separate complete Ingest buttons are rendered at the end of the workflow to trigger actions specific to local upload or dataONE download workflows. These buttons are rendered programmatically depending on the state of the selectInputMethod radio button.
209 - Converted time inputs to properly merge startDate and startTime with EndDate and EndTime so they can be inserted into the start_date and end_date columns in BETYdb.
210 - Error handling introduced using `shinytoastr` package
211 - DESCRIPTION: `Depends`: PEcAn.visualization, shinytoastr, shinyWidgets, shinyjs
212
213- pecan/base/db
214 - New File: `input.format.vars.R`. This function registers the format and the (optional) formats_variables record using `db_merge_into`.
215
216- `data.atmosphere`
217 - `check_met_input_file` -- Check that target met file conforms to PEcAn meteorology data standard.
218 - `get_cf_variables_table` -- Retrieve CF variables table as a `data.frame`
219
220
221- docker:
222 - Added updated docker container builds
223 - Use docker.sh to create docker images
224 - Use release.sh to push released images to push to docker registry (hub.docker.com by default)
225 - Create pecan/depends docker image that holds all PEcAn dependencies
226 - Needs to build seperatly, not part of the docker.sh build process to speed things up
227 - Build using `(cd docker ; docker build -t pecan/depends:latest -f Dockerfile.depends .)`
228 - docker-compose.yml file to bring up full PEcAn stack using docker
229 - First time to start requires to install BETY database (see documentation)
230 - SIPNET docker image which works with PEcAn docker stack
231 - Data container that will download and install demo data in /data folder
232
233
234### Removed
235 - pecan.worldmap function no longer used, dropped from visualization package
236 - shiny/Data-Ingest/DESCRIPTION no longer `DEPENDS` on `shinyFiles` or `shinycssloaders`
237
238### Changed
239- PEcAn.utils functions run.write.configs and runModule.run.write.configs have been moved to PEcAn.workflow. The versions in PEcAn.utils are deprecated and will be removed in a future release.
240- Fixed Git instructions and remote execution instructions.
241- Five functions from PEcAn.utils functions have been moved to other packages. The versions in PEcAn.utils are deprecated, will not be updated with any new features, and will be removed in a future release.
242 - run.write.configs and runModule.run.write.configs have been moved to PEcAn.workflow
243 - read.ensemble.output, get.ensemble.samples and write.ensemble.configs have been moved to PEcAn.uncertainty
244- Change the way packages are checked for and called in SHINY apps. DESCRIPTION files in SHINY apps are not the place to declare pacakge dpendencies.
245
246## [1.5.3] - 2018-05-15
247
248### Fixes
249- read.output now accepts date-times for start.year and end.year argument (#1887)
250- read.output no longer assumes timestamps are in days since the beginning of the year
251- Fixed xss issue in setup folder, now require users to login before accessing the setup folder scripts.
252- Fixed issue where in website not all sites are shown #1884
253- Fixed status page, should now be run from cronjob, creates static page
254- Fixed bug that overwrote remote met file paths with local file paths
255- PEcAnRTM:
256 - Remove non-portable extensions from `src/Makevars`. This should make the package Windows-compatible.
257- Fixed BrownDog shiny issues of removing site without geometry; fixing sites.length==0; removing old map markers when change input$type; fixing agreement bug when change input$type
258
259### Added
260- Functionality to read pft-specific outputs and to run sensitivity analysis on pft-specific outputs.
261- Ability to allow for insecure sync using -k flag
262- Added information on how to join slack
263- PEcAn.BIOCRO now supports BioCro version 1.0. BioCro 0.9x models should still work as before, but note parameter and weather format changes in the `Changed` section below.
264- Added new model package (PEcAn.dvmdostem) and initial wrappers for integration of the DVM-DOS-TEM model and tested.
265- PEcAn now supports PFTs whose members are cultivars rather than species, and will automatically restrict the meta-analysis to matching records, e.g. runs with a PFT containing only Panicum virgatum 'Cave-In-Rock' will not use observations from Panicum virgatum 'Alamo', but a PFT containing the whole species will use observations from both. However, there is not yet any BETYdb interface to *create* cultivar-PFTs other than manual SQL.
266- New base package `PEcAn.workflow`, for functions used to perform the each major step of the analysis. These were previously scattered in other base packages.
267- Added PR review time estimate to PR template
268- New set of `PEcAn.logger` functions similar to `stopifnot` to facilitate assertive programming: `severeifnot`, `errorifnot`, `warnifnot`, `infoifnot`, `debugifnot`
269- PEcAnRTM:
270 - Exposed PROSPECT absorption coefficients and `gpm()` function ("generalized plate model"), facilitating experimentation with different absorption coefficients
271 - Added `spectra` S3 class and methods for subsetting (e.g. `myspec[[400:700]]`), plotting (`plot()` and `matplot()`), and combining spectra by wavelength.
272 - Added `resample` functions for quickly resampling spectra (and, more generally, vectors and functions) to different dimensions.
273 - `EDR` API has been revised. Setup has been refactored from EDR via new `setup_edr` function, which relies on the ED utilities (see `PEcAn.ED2` below), and the `EDR` function now focuses only on execution. Also, added new `params2edr` function to make it easy to convert complex EDR parameters list to flat parameter vector required by `invert_bt` (or other optimization functions).
274- PEcAn.ED2:
275 - New set of utilities for working with ED meteorology and vegetation inputs, and the ED2IN file. Existing PEcAn code has been revised to use these utilities.
276- PEcAn.data.atmosphere:
277 - New utilities for efficiently downloading NARR time series using THREDDS/OpenDAP
278
279### Removed
280- Removed deprecated copies of PEcAn.utils::SafeList, PEcAn.utils::listToXml (both moved to PEcAn.settings in v 1.5.2), and PEcAn.utils::fqdn (moved to PEcAn.remote in 1.5.2). This fixes the masses of deprecation warnings in otherwise normal run logs (#1719).
281
282### Changed
283- Updated wrappers for FATES model to work with recent CLM5 release. Updated write.config, job.template, and other associated files to work with CLM5 inputs and met drivers
284- Updated model2netcdf.MAAT to use ncdf4::ncvar_def to define netCDF variables
285- Fixed an remote code execution discovered by NCSA security team.
286- Column name changes for newly generated biocromet csvs: `SolarR` is now `solar` and `WS` is now `windspeed`. Previously generated met files with the old names will still work for BioCro 0.9 runs, but will need to be renamed before using them with BioCro 1.0.
287- write.configs.BIOCRO now requires a model version specification so that it can format parameters to match your version of BioCro. Set it in your Bety model record or in the model$revision field of your pecan.xml.
288- When using BioCro 1.0 and no parameter file is given, PEcAn will make an attempt to find default parameters for your genus in the datasets provided by the BioCro package. Note that the default parameter files provided in `models/biocro/inst/extdata/defaults` will *not* work when using BioCro 1.0.
289- Added documentation how to submit a run from the command line
290- Updated models/maat to provide support for latest model code updates
291- PEcAn.DB function `rename.jags.columns` renamed to `rename_jags_columns` to avoid conflict with S3 method naming conventions
292- Replaced `rhdf5` library with `hdf5r`, a more modern alternative that is available on CRAN.
293- PEcAn.DB function `runModule.get.trait.data` has been moved to the new PEcAn.workflow package to avoid a circular package dependency between PEcAn.DB and PEcAn.settings.
294- Major documentation refactoring. The documentation names are now directly tied to the order in which they are rendered, and all `Rmd` files in all subdirectories of the documentation source are rendered by default. The overall structure of the documentation has been revised for clarity and cohesiveness.
295- Edited met2model.ED2 to not enforce leap years.
296- Integrate demo 1 into basic user guide
297
298## [1.5.2] - 2017-12-07
299
300### Fixes
301- Updated models/ed/data/pftmapping.csv to include two new BETYdb PFTs
302- Simple fix to models/ed/R/write.configs.ed.R to properly align pss and css file prefix
303- Fixed issue #1752 by updating the site.lst() function to include `site.id=site$id` instead of site.id=site, as site is an object not just the id
304- Update to PEcAn.ED2::met2model.ED2 to fix issue with rhdf5::h5write. Bug fix to #1742
305- Fixed write.config.xml.ED2 parsing of data/history* files
306- `PEcAn.utils` now lazy-loads data for faster execution of functions that consult lookup tables, especially `to_ncvar`.
307- Fixed incorrect `PEcAn.BIOCRO` daily and yearly results: Was calculating every row from whole simulation instead of that day (#1738)
308
309### Added
310- New Dockerfile to create PEcAn specific container for SIPNET.
311
312### Removed
313- Removed `PEcAn.utils::model2netcdf`, which has been deprecated since PEcAn 1.3.7. Use `model2netcdf.<YOURMODEL>` in the appropriate model package instead.
314
315### Changed
316- Major namespace cleanup in the `PEcAn.utils` package. It now loads more quietly and is much less likely to mask functions in a package you loaded earlier.
317- Moved many functions from `PEcAn.utils` into other PEcAn packages. The `PEcAn.utils` versions still work with a deprecation warning, but will be removed in next release.
318 - `listToXml` and `SafeList` moved to `PEcAn.settings`
319 - `fqdn` moved to `PEcAn.remote`
320- PEcAnRTM: Removed effective sample size normalization from likelihood calculation. It was giving weird results.
321
322## [1.5.1] - 2017-10-05
323
324### Fixes
325- Fixed hyperparameter draws in PDA
326- Show workflowid in the URL when run is finshed and user clicks results (#1659)
327- `PEcAn.BIOCRO` now uses PEcAn-standard variable names. As a result, two output variables have been renamed but keep their existing units and definitions:
328 - `StemBiom` renamed to `AbvGrndWood`
329 - `RootBiom` renamed to `root_carbon_content`
330- Improved make install logic (#1558)
331- Fixed remote code execution #1545
332- Added check for NA end/start year in read.output
333- Fixed jagify bug for raw field data
334- Fixed bug (order of dims in nc_create) introduced in model2netcdf.DALEC by standard_vars changes
335- Cleaned up NAMESPACE and source code of `PEcAn.DB` (#1520)
336- Debugged python script in call_MODIS in data.remote to allow MODIS downloads
337- Fixed FATES build script to work on ubuntu
338 SIPNET output netcdf now includes LAI; some variable names changed to match standard
339- Cleanup of leap year logic, using new `PEcAn.utils::days_in_year(year)` function (#801).
340- Replace many hard-coded unit conversions with `udunits2::ud.convert` for consistency, readability, and clarity
341- Refactored extract_soil_nc to create soil2netcdf, which will write soil data out in PEcAn standard.
342- Added a new retry.func() to base/utils to provide ability to re-try a function X times before stopping. Currently using this function in the download.CRUNCEP() function to handle slow responses from THREDDS.
343- Reformatted call_MODIS netcdf output to reflect the orientation of the MODIS grid
344- Remote execution is more robust to errors in the submission process, not just the actual model execution
345- PRELES model run script bug fix
346
347### Added
348- Functionality to pass hyperparameters via settings
349- Created new (and very rudimentary) web interface for downloading data from the dataone federation into the PEcAn database. More updates to come.
350- Expanded initial conditions workflow for pool-based models, including PEcAn.data.land::prepare_pools to calculate pools from IC file (to be coupled with write.configs)
351- New `PEcAn.utils::days_in_year(year)` function that should make it easier to work with leap years.
352- New `PEcAn.data.atmosphere::solar_angle` function that replaces math that occurs in some models.
353- New `PEcAn.benchmarking::align_pft` fucntion that aligns data assosiated with two different plant functional types
354- #1594 shiny/workflowPlot Adding interactiveness using ggploltly
355- #1594 shiny/workflowPlot Load outputs from multiple runs of the model
356- #1594 shiny/workflowPlot Ways to toggle geometries (e.g. geom_point vs. geom_line).
357- #1594 shiny/workflowPlot Smoothing using geom_smooth (Slider for specifying moving window width)
358- #1594 shiny/workflowPlot Comparing model output vs loaded data according to [tutorial](https://github.com/PecanProject/pecan/blob/develop/documentation/tutorials/AnalyzeOutput/modelVSdata.Rmd)
359- Allow SIPNET and DALEC met files and model2netcdf to start or end mid year
360- A Pre-release database clean up script that deletes unused/unassosiated entries from the database
361
362### Changed
363- Clean up directory structure:
364 * Move `base` packages (`utils`, `settings`, `db`, `visualizaton`) to a `base` directory, for consistency with `modules` and `models`
365 * Move `logger.*` functions out of the `PEcAn.utils` package and into the `PEcAn.logger` package
366 * Move `remote` functions out of the `PEcAn.utils` package and into their own `PEcAn.remote` package.
367- #1594 shiny/workflowPlot Refactoring of code. `get_workflow_ids` in db/R/query.dplyr.R changed with `ensemble = FALSE`. Also allowing to load all workflow IDs. `load_data_single_run` and `var_names_all` also moved from shiny/workflowPlot/server.R to query.dplyr.R
368- `PEcAn.remote::start.model.runs` has been significantly refactored to be less redundant and more robust
369- `betyConnect` function in `query.dplyr.R` is now refactored into `read_web_config` so that the the Data-Ingest app can leverage `read_web_config` and provide it with a machine specific filepath for `.../dbfiles`
370- Rpreles and Maeswrap package moved to suggest checked for within package function.
371
372
373## [1.5.0] - 2017-07-13
374### Added
375- Added cron job and script for the sync of the database.
376- Added PEcAn.utils::download.file() to allow for use of alternative FTP programs
377- Updated downloadAmeriflux and downloadNARR to make use of PEcAn.utils::download.file()
378- Added -w flag to load.bety.sh script to specify the URL to fetch the data from
379- Add new table sites_cultivars to betydb sync scripts (dump and load)
380- Added docker container scrips (.yml) to create docker container for PEcAn
381- Added the configuration edit page to allow easy modification of config via web interface
382- Thredds server documentation and catlog generating script
383- Added new standard variables table (standard_vars.csv) and to_ncvar and to_ncdim functions in PEcAn.utils
384- Added initial conditions file io functions for pool-based models in data.land
385
386### Changed
387- upscale_met now accepts ~any valid CF file (not just full years), retains correct time units, and respects the previously ignored `overwrite` parameter
388- Better date handling in BioCro functions
389
390## [1.4.10.1] - 2017-04-18
391
392### Changed
393- Bugfix in Fluxnet2015
394- Update Git workflow in Documentation
395- download.CRUNCEP now uses CF-compliant time units (days since start of year instead of "secs")
396- Bugfixes in met.process
397
398
399## [1.4.10] - 2017-03-27
400Documentation
401
402### Added
403- Source Rmarkdown and deploy scripts for PEcAn documentation
404- Autocorrelation correction in PDA and scaling factor in emulator
405
406### Changed
407- now dumping/loading experiments, cultivars_pfts, current_posteriors, experiments_sites experiments_treatments, trait_covariate_associations [BETY #403](https://github.com/PecanProject/bety/issues/403) [BETY #491](https://github.com/PecanProject/bety/issues/491)
408
409### Removed
410- Ameriflux is no longer selectable from the web gui [#1291](https://github.com/PecanProject/pecan/issues/1291)
411
412## [1.4.9] - 2016-12-10
413Benchmarking, code cleanup
414
415### Added
416- benchmarking code
417
418### Changed
419- no more build.sh, using Makefile
420- Lots of code cleanup thanks to @bpbond
421
422## [1.4.8] - 2016-08-11
423Camp PEON: Assimilation, multi-site, soil params, Maespa, LPJ-GUESS, improvements to web & tutorials
424
425## [1.4.7] - 2016-07-13
426CMIP5, Shiny, FLUXNET2015, Global Sensitivity
427