The operations that involve fitting many PARAFAC models
(feemsplithalf and feemjackknife) now handle the
arguments parallel and cl on their own. If parallel
processing is enabled, they schedule the calls to
feemparafac(...) on the cluster. This makes it possible to
run more than nstart (10 by default) models at the same time
and avoid the wait for one remaining random start before moving on
to the next model. Due to implementation differences, this also
avoids the need to call clusterEvalQ(., library(multiway))
before performing parallel processing.
The default stopping criterion in feemparafac has been
changed from 10^{-4} to 10^{-6}. The
previous value caused misleadingly pessimistic split-half results
for real world datasets, failing the comparisons due to the models
being underfitted.
The feemscale function now raises an error when the
returned scale factor turns out to be NA.
The plot.feemparafac method used to sort the components
lexicographically, making it inconvenient to read plots for models
with more than 9 factors. The problem is now fixed, with
coef(type = 'surfaces') returning the data.frame used
to construct the plot.
The feem(..., format = 'table') method now treats the
dec argument specially, making it possible to read FEEM
tables with commas as decimal separators.
The coef.feemsplithalf(kind = 'factors') method now
avoids repeatedly calling feemcube.feemparafac(), which is
expensive for large feemcubes due to having to subset them.
This speeds up all related plots.
New split-half plot and coefficient output,
kind = 'aggtcc', taking the minimal TCC values over
individual components and plotting their distributions as a boxplot
combined with a violin plot.
New split-half plot and coefficient output,
kind = 'bandfactors', summarising the distributions of the factor
value over the different subsets as estimates and bands.
The operations that involve fitting many PARAFAC models
(feemsplithalf and feemjackknife) now adjust the signs
of the loadings to be as positive as possible (using
resign(absorb = 'C')) before matching the components. This
makes it possible to validate models with loadings not constrained
to nonnegativity.
The feemparafac function now returns the time it took
to compute the decomposition in the time attribute of the
object. Depending on whether a parallel cluster or a parallel
BLAS has been used, most of the time may be spent in elapsed
(waiting for the cluster), user.self (computing in more than
one thread), or even sys.self (BLAS threads spinning in
sched_yield()).
New function feemcorcondia to perform the core
consistency diagnostic (Bro & Kiers, 2003) on PARAFAC models, with
added support for missing data.
The feemflame function gains arguments
control.parafac and control.cmf, making it possible to
adjust the termination criteria of the underlying methods.
The previous version of the package was accidentally broken on
versions of R where apply() did not yet have a
simplify argument. More strict release testing procedures
will be applied in the future.
New argument like to reorder.feemparafac and
rescale.feemparafac makes it possible to order and scale the
components to match another model. The package now re-exports the
rescale generic from the multiway package.
Fixed an error in feemscatter(method='whittaker') where
the nonneg parameter could be ignored (turning the
non-negativity protection off) if the wavelength grid was not sorted.
This mistake has been present since version 0.3-0, where Whittaker
smoothing had been introduced.
Inverted the luminance of the diverging.colours palette,
setting it to brighter for lower-magnitude values. The signs of the
residuals are much easier to understand now.
The feemcube.feemflame method is now actually exported.
The plot() method for feemflame objects now
correctly forwards its arguments to lattice functions for
type = 'both'.
New function absindex computes absorption coefficients,
absorbance values, spectral slopes and their ratios.
When importing files with a non-default encoding,
feem.character now avoids leaking a connection per file. (They
were cleaned up anyway by the garbage collector, but with a warning.)
The custom progress bar in feemflame is now re-drawn
correctly on R GUIs.
On R ≥ 4.2, equations in HTML help are typeset from LaTeX source, not manually translated HTML.
t() on feem objects now results in an unclassed
matrix, not a broken feem object.
feemscatter now accepts scattering bands with asymmetric
widths. On some fluorometers, this is needed in order to remove
unwanted signal in anti-Stokes zone.
New palette, diverging.colours, is used to plot residual
objects.
The const argument of feemparafac (and other
functions that call it) now defaults to non-negativity. The previous
default of an unconstrained model can be achieved by setting the
argument to NULL.
Fixed the bugs in feemjackknife and feemsplithalf
which prevented them from working with 1-component models.
Fixed the bugs preventing feem and feemcube
objects from being properly indexed by their dimnames.
The data(feems) dataset has been replaced with a
synthetic one, with perfectly matching absorbance spectra.
New function feemflame to model fluorescence and
scattering signal at the same time.
New function feemindex to calculate fluorescence indices
and named peak values used in dissolved organic matter analysis.
When the samples and the absorbance information don't match,
feemife now provides an explanation of what went wrong.
When an error happens during the processing of a feem
inside a collection (a feemcube or a list of feems) or
during feemlist file import, the offending sample (or file) is
now reported, and the error can be trapped without losing the
traceback.
The feemcube function can now return the original cube
used to fit feemparafac, feemjackknife,
feemsplithalf objects.
feem now supports text files from Edinburgh Instruments
F900 software.
feemlist can now take an importer function as an argument.
feemsplithalf can limit the number of split-combine
comparisons (the splits argument) or take a fixed list of
halves to fit and compare (the fixed argument).
HTML version of the manual now passes Tidy validation without warnings.
Fixed a bug in numerical differentiation code for
feemscatter(method = 'whittaker') that resulted in d-th
order derivatives being 2^d times smaller than they should have
been. Existing optimal lambda values will have to be updated by
dividing them by 2^{2d}.
When interpolating using “loess” or “kriging” methods, predicted negative values are replaced with zeroes. Previously, this happened for the whole FEEM, but only with “loess”.
feemcube now can also extract the original cube from
feemparafac objects.
Simplified the feemlist(..., simplify.names = TRUE) logic,
hopefully preventing the test failure on some macOS machines.
Made the feem(..., 'table', fileEncoding = '...') logic
more robust, especially on older versions of R where it could still
fail when fileEncoding could not be represented in the locale
encoding.
example(write.openfluor) no longer leaves files it creates
in the per-session temporary directory.
Added a new ‘whittaker’ interpolation method for
feemscatter and feemgrid based on Whittaker smoothing
with configurable difference order(s) and optional iterative penalty
against resulting negative values. This brings in a dependency on the
recommended Matrix package.
The ‘pchip’ interpolation method now offers a choice whether to interpolate the EEM by row, by column, or to do both and average the results (thanks staRdom for the trick). It also doesn't crash anymore when trying to 1-D interpolate less than 3 defined points, falling back to linear interpolation instead.
Added feem methods for strings and connections, making
it possible to read EEMs from files in simple table formats, including
those produced by Horiba and Panorama software.
Added the feemlist generic to construct lists of
feem objects from eemR or EEM objects or vectors
of file and directory paths.
Added the groups argument to feemsplithalf,
making it possible to stratify the resulting halves according to a
given factor value.
Added the feemgrid generic to interpolate FEEM(s) on a
given wavelength grid.
Added coef methods for feemparafac,
feemsplithalf, feemjackknife returning the coefficients
and summary values (e.g. TCC for split-half) in “long”
data.frame format.
Added function write.openfluor to export a fitted
feemparafac object for the OpenFluor service.
Equations in the HTML help pages are now typeset using HTML features instead of plain ASCII.
Dependency on R >= 3.3 is now acknowledged. The
package doesn't actually pass R CMD check on prior versions
of R. Patches to lower the required R version are welcome if they
don't result in extensive rewrites.
Other documentation improvements.
Fixed a serious bug in feemsplithalf that would result
in partially overlapping splits being compared. It has been present
since v0.1-1. I apologise to any users of albatross who found
out they have been bitten by this. Steps have been taken to make sure
that other bugs like this don't still lurk in the code and to prevent
similar bugs from occurring in the future.
Potentially breaking change: feemife now accepts either
named absorbance / optical path length data exactly matching unique
names of samples, or a collection of exactly same length with either
names missing. Named collections of the same length as number of
samples which don't contain all unique sample names are going to be
rejected from now on.
Saved feemsplithalf and feemjackknife results now
should consume much less space and much less memory when loaded back.
This is achieved by saving the FEEM cube object in an environment and
only referencing it using vectors of indices instead of keeping copies
of its subsets.
New interpolation method: “kriging”. Very
resource-intensive, but results can be nicer-looking than either
loess or pchip.
More consistent progress bars for operations on collections of
feems. Scatter correction can now be run in parallel.
Minor documentation improvements.
Fixed a bug in feemjackknife and feemsplithalf,
which could manifest when rescale argument is not 3 and
loading scales are wildly different, resulting in poor approximation
for some of the modes.
Made the documentation for PARAFAC/split-half/jack-knife plots more prominent.
Made tests and examples much faster.
Made the gamma argument in marine.colours much
more useful.
Minor improvements in documentation.