Fixed bug in MAPD calculation by copynumber workflow. In 1.10.0, the MAPD values were computed before putting probesets in genome order. The implication is that MAPDs reported by 1.10.0 were inflated relative to the real MAPD values.
Add genome GC content correction to copynumber workflow.
Add a log file to apt-summary-genotype
Add on-disk mode to apt-cel-extract (on by default)
Add progress meters for chip layout file reading (ie CDF)
Fixed bug where block size and memory check code was still being run even when on-disk mode was set. The implication in 1.10.0 was that multiple iterations (library file reads) were being done unnecessarily. Results were still correct.
Add copynumber binaries to windows installer (missing in 1.10.0 release)
Changed the code to only read lib files only once when doing one iteration in apt-probeset-genotype and apt-probeset-summarize.
Clean up of a5 options for apt-probeset-genotype
Add a5 options to apt-probeset-summarize
Refactored the analysis engines to inherit from a common engine base class.
Version:
apt-1.10.0
Date:
Wed Jul 9 2008
Fixed bug where pm-sum method was reporting values equivalent to pm-only when using SPF file rather than a CDF file. This is in the context of using apt-probeset-summarize with genotyping array via the "expr" addition to the analysis string.
Fixed bug where incorrect "x" coordinate was reported in feature effect output files from apt-probeset-summarize and apt-probeset-genotype (all releases) and in apt-cel-extract (releases newer than 1.8.0) output. Incorrect values only show up when working with arrays where the number of rows and columns in the CEL file are not equal.
Fixed bug in PLIER where feature effects were updated after the last update to target responses
Added copynumber functionality for SNP6 (see apt-copynumber-workflow).
Added Canary copynumber calling for SNP6 (see apt-canary).
Complete refactor of apt-chp-to-txt. Now handles most chp file formats and types. Resequencing and Tag chp files are not handled by this tool.
Added Vignettes:
SNP6 Copynumber analysis using individual components
Single sample genotyping (ie for QC purposes)
Removed apt-cel-to-pq. Supplanted by apt-cel-extract.
Misc
Added experimental HDF5 output file formats to apt-probeset-genotype
Added on-disk intensity mart. Set as the default for apt-probeset-summarize and apt-probeset-genotype.
Fixed memory capping on x64 windows and 64-bit OS-X.
Added the ability to apply pre-computed feature effects when using median polish in apt-probeset-summarize and apt-probeset-genotype.
Added an adaptor type normalization (used by copynumber workflow)
apt-probeset-genotype will now report signal estimates for copy number probesets when the --all-types and --summaries option is given
Version:
apt-1.8.6
Date:
Mon Feb 3 2008
Fix bug in IterPlier where one would get a segfault if the number of chips exceeded 100.
Change from float to double in QuantBirdseed methods to improve cross platform portability. This may result in slight differences in the confidence value and in rare cases a change between making a genotype call versus a no-call.
order of results in apt-probeset-genotype report file have changed due to some refactoring of the code.
added birdseed-v2
added ContrastQC qc metric to apt-geno-qc
fix memory estimation (don't cap at 2G on Windows (x64) and OS-X (ppc64, x86_64)
apt-cel-transformer: remove ability to change formats. add ability to handle agcc files
Improvements to command line option handling
Allow med-norm to be followed by other chip stream components when using a fixed target
added a target and doavg options to quant-norm
fix bug in --select-probes option of apt-probeset-genotype where probe IDs were not reported correctly
Version:
apt-1.8.5
Date:
Wed Oct 31 2007
Change to the DM Implementation DM calls/confidences can be reported by apt-geno-qc and are generated and used by apt-probeset-genotype to generate seed calls for BRLMM on 500k data. Differences were observed between different platforms (particularly 64-bit vs 32-bit) in a small number of SNPs (ie 0-5 SNPs out of 250,000 on 500K Sty Chip). Code changes were made which rectify this, however you may see some some changes in DM results because of this.
Binary Releases
Added 64-bit Windows Binaries (x64)
New documentation
Vignettes
Masking Out Probes in apt-probeset-summarize
Use of APT to Analyze WT Based Exon and Gene Expression Arrays
Added new top level pages and new navigation for end user (binary release) documentation
Change Log
Vignettes
FAQs
License
File Formats
Platforms
Updated Manuals
Added new FAQ items to application specific manuals
New Applications:
apt-cel-convert: a tool for converting between various CEL file formats.
Changed Applications:
Most Applications
log messages on stderr rather than stdout
apt-cel-transformer:
support PGF/CLF and SPF in addition to CDF
apt-cel-to-pq:
support AGCC CEL files
apt-cel-extract:
added support for AGCC CEL files
complete rewrite of this application to use the core APT ChipLayout, ChipStream, and SparseMart framework. As a result this is really a new tool as far as command line arguements and behavior of the application are concerned.
the tool can now scale to a large number of cel files (although you will need to set the --block-size manually)
block number is reported (critical for assigning genotype probes to alleles)
you can provide an analysis string (including pm-adjustor) to manipulate probe intensities
you can report just the pm values, the pm and background (ie mm, gcbg, ...) values together (ie 132,22), or the result of the pm minus background
you can provide both a probeset list and probe list (intersection will be extracted)
probeset and probe lists now work with CDF files
the probeset type is reported (useful for multi-type CDF files like SNP6)
the tool will work without any library files (with reduced functionality of course)
enable use of experimental spf library files
apt-geno-qc:
added --chrX-probe and --chrY-probe options which enable CN probe gender calling
added --probe-class-file option which will turn on the CelStatListener and report probe intensity means by group
new column headers -- consistency with apt-probeset-genotype report.
new ordering of columns -- this will continue to vary over time. You should rely on column labels rather than column position to parse out specific information. Same holds true for apt-probeset-genotype report file
dropped --rpt/-r option, (replaced with --dm-out option)
Added --dm-out, --dm-thresh, and --dm-het-mult options for enabling DM calls and confidence output and setting DM options. Note that the output is now in TSV format.
complete refactor of the source code
apt-probeset-summarize:
Fixed a bug in MAS5 background correction routine when using pgf file. In short, the number of probe sets was set to 0 when using a pgf file.
quality assessment metrics and QC probeset values are now written to the AGCC CHP file headers
probesets which failed to be quantified (ie certain controls) are no longer written to the AGCC CHP file. In previous versions such probesets would typically show up with a 0.0 quantification value.
changes to how the md5sum (gc background probes and normalization probes) is computed to ensure cross platform consistency
syncronized meta header data between apt-probeset-genotype and apt-probeset summarize. IE, there will be some changes to the meta info going into CHP files and other output files.
added support for SPF files
fixed bug where mas50-bg chipstream failed if there were additional chip streams after it
added additional boundary conditions in RMA::findBandWidth in order to match behavior of BioConductor. May fix errors with certain chips with low overall intensity.
previous behavior was to not output mm_* quality metrics when bgrd probes were provided (via --bgp option). Now mm_* is reported if there are more than 5000 MM probes listed in the library file(s). Note that for the Human, Mouse, and Rat Exon 1.0 ST arrays, this will be dominated by the "genomic background" probes which are flagged as MM probes in the PGF file.
removed the path from the cel file names listed in the report file.
improved masking of probes (still experimental)
--rm-probes to --kill-list (sync w/ apt-probeset-genotype)
take into account probeset when masking probes -- ie you can now mask out a probe that is in more than one probeset
masking file can have probe_id (1 base) or x/y position. if both are present it will check to see that they are consistent.
when probesets and meta probesets are completely masked out, a warning will be issued rather than having the software exit with an error message.
added --mem-usage option to set memory use in MB as alternative to setting block-size
added --chip-type option. can be specified multiple times. This option will dictate the chip type in the output files and it will also specify what chip types the cel and lib files are checked against. Note that this will override the typcial check between chip types in lib files versus those in the cel files.
When doing CDF based analysis, chip types based on substrings ending with "." are also considered a match. IE, CDF file foo.bar.v1.r10.cdf will be detected as matching the possible chip types foo.bar.v1.r10, foo.bar.v1, foo.bar, and foo.
new --cc-md-chp-output option will generate experimental AGCC Multi Data CHP files.
added a --set-analysis-name option which will override the name computed from the analysis string. This option will affect the prefix used for output file names and the labels in certain meta fields and/or report file metrics.
apt-probeset-genotype:
fixed bug where chip-type in CHP file header was not set when using --force option
removed --txt-output option, which had a bug where calls were not thresholded in text output file for birdseed
fixed bug on solaris where use of birdseed method would cause a segfault
Added documentation on --write-models option of apt-probeset-genotype
fix reported analysis spec for all genotype calling methods. The genotype calling method (Quant Method) was not reported and in some cases incorrect algorithm parameters were reported.
for BRLMM-P, cluster_distance_mean metric is now correctly reported. Previously it was just set to zero.
fixed bug where -s option with --cc-chp-output would fail
added a number of additional metrics to the report file:
DM call rate (when DM listener is run -- ie BRLMM)
Various gender calls based on methods which are suitable to the chips/analysis being run
Various metrics underlying the gender calls
added a --probe-class-file option which will read in a file grouping probes into classes for use by the CelStatListener.
added --read-genders option to read genders from a file.
implement --set-gender-method option to force use of a particular gender method. Current options are:
"cn-probe-chrXY-ratio"
"dm-chrX-het-rate"
"em-cluster-chrX-het-contrast"
"user-supplied"
"none"
"supplied-genotypes-chrX-het-rate"
change chrXForce to --no-gender-force: ie run even if no suitable gender method is available
removed labelz (brlmm-p) KXY option which did nothing.
added birdseed-dev genotype calling method. "birdseed-v1" (alias for full analysis, and quant method name) will return the original birdseed implementation. "birdseed-dev" (alias for full analysis, and quant method name) will return the latest integrated method for birdseed from the Broad. This implementation will change over time. "birdseed" (alias for full analysis, and quant method name) will return the original birdseed-v1 implementation.
An option, --set-analysis-name, will be added to apt-probeset-genotype. This will allow one to set the analysis name irregardless of what analysis method is used. IE, "--set-analysis-name %birdseed" would allow birdseed-dev to masquerade as birdseed. Note that the analysis spec field would still enumerate the exact method used.
birdseed-dev has been updated to reflect birdseed version 2.6 from the Broad.
a column header for the cel file names is now present in report file output. Meta information is now included in the report file header. Additional column (call_rate) was added which matches that in the AGCC CHP file header. Also note that only the filename is now listed. The path, if present is removed.
changes to how the md5sum (gc background probes and normalization probes) is computed to ensure cross platform consistency
syncronized meta header data between apt-probeset-genotype and apt-probeset summarize. IE, there will be some changes to the meta info going into CHP files and other output files.
added support for SPF files
improved masking of probes (still experimental)
see details under apt-probeset-summarize
currently cannot remove an entire probeset -- will result in an error. use probeset list with -s option instead.
added --chip-type option. can be specified multiple times. This option will dictate the chip type in the output files and it will also specify what chip types the cel and lib files are checked against. Note that this will override the typcial check between chip types in lib files versus those in the cel files.
When doing CDF based analysis, chip types based on substrings ending with "." are also considered a match. IE, CDF file foo.bar.v1.r10.cdf will be detected as matching the possible chip types foo.bar.v1.r10, foo.bar.v1, foo.bar, and foo.
added a --set-analysis-name option which will override the name computed from the analysis string. This option will affect the prefix used for output file names and the labels in certain meta fields and/or report file metrics.
Source Code
refactor how genders are handled. Now GenderCalls interface which the various gender calling methods implement. A map of GenderCalls is then generated and passed around in the genotype engine.
refactor EM gender calling to implement GenderCalls and ChipSummary, hook ChipSummary into reporters
refactor DM listener to implement GenderCalls and ChipSummary, hook ChipSummary into reporters
refactor PgOptions class to dynamicly allocate options. (See apt-probeset-genotype for example of the new interface.)
integrated birdseed 2.4 from the Broad
QuantMethods: birdseed-v1, birdseed-dev, birdseed (which is birdseed-v1)
Analysis Strings: birdseed-v1, birdseed-dev, birdseed (which is birdseed-v1)
refactor the CN probe gender calling method as a CelListener, GenderCalls, and ChipSummary. Hook ChipSummary into reporters.
change eps/epsilon alg params to use full number, not scientific notation which breaks the state propigation in analysis-spec
moved SpecialSnps into chipstream folder
fixed propigation of user set values into reported analysis spec for birdseed
current birdseed folder renamed birdseed-dev
created birdseed-v1 folder with previous birdseed implementation [GenderCaller and FitSNPGaussiansPriors3]
The code in birdseed-v1 moved into a C++ namespace "birdseed::v1".
The code in birdseed-dev moved into a C++ namespace "birdseed::dev".
New analysis spec alias: birdseed-v1, birdseed-dev. birdseed-dev will always point to the latest Broad recomendations. birdseed-v2 will be added when the birdseed v2 method is frozen.
Analysis spec "birdseed" would remain as a pointer to "birdseed-v1".
birdseed correction-factor was locked at 1. can now be set by user.
implement FileGenders for reading genotype genders -- also checks against cel file names in opts.celFiles (previous stub did not).
removed unused legacy code in ChpCheck.h and CalvinChpCheck.h
fix memory allocation in Pg::Options
More checks regarding GC count info when using gc-bg, pm-gcbg, and dabg
More use of Verbose class for log output (rather than using cout or cerr directly.
More use of Err class (rather than exit() or throwing exceptions)
Verbose::out and Verbose::warn no default to cerr rather than cout.
Moved chip-type checks out of the command line wrappers and into the engine code. (ProbesetSummarize and ProbesetGenotype)
generally you do not want to fill in the chipType and chipTypeSupplied members of the engine options class.
setting chipType will force all output to be reported as that chip type.
setting chipTypeSupplied will force chip type checking against that list of chip types. If chipType is not set, it will be set to the first entry in chipTypeSupplied.
Refactored how chip summary metrics are propigated into output files. There is now a ChipSummary interface which various classes can implement. These classes can then be registered with certain reporters (ie Run Report and AGCC CHP reporters) as sources of chip summary info.
Version:
apt-1.8.0
Date:
Tue May 29 2007
New Applications:
apt-geno-qc: a tool for computing single chip QC metrics for WGSA (500k, Genome Wide SNP5, and Genome Wide SNP6) genotyping chips
Changed Applications:
apt-probeset-summarize:
Fixed overflow error in accumulator for DABG values (previous behavior affected probesets with lots of probes where the DABG p-value with go to 1 rather than 0.
fixed bug related to non-square CEL files where the mapping between x/y coordinates and probe IDs was incorrect
atom_id field in the feature response and feature residual files now contain "0". Atom IDs are no longer retained in the analysis software in order to reduce the memory requirements. As a result, the information is not available for reporting into this output file.
added ability to treat genotyping and copy number probesets as expression probesets. See "--explain expr".
added pca-select and spect-select feature selection methods (intended for use in computing gene level estimates from WT Gene and Exon arrays). See "--explain pca-select" and "--explain spect-select".
enable use of meta probeset files (mps) with cdf files
added PM+MM (pm-sum) adjustor. See "--explain pm-sum".
added median and mean summarization methods. See "--explain median".
apt-probeset-genotype:
fixed bug related to non-square CEL files where the mapping between x/y coordinates and probe IDs was incorrect
added birdseed method for genotyping calling (associated with SNP6)
added brlmm-p-plus method for genotyping calling (associated with SNP6)
added brlmm-p method for genotyping calling (associated with SNP5)
Source Code
reduce memory requirements of apt-probeset-summarize and apt-probeset-genotype by using a new internal structures for chip layout information
added support for an experimental chip layout format (SPF)
new build system with multiple static libraries
Version:
apt-1.6.0
Date:
Fri Sep 29 2006
General
Upgraded to Newmat 10D
Added preliminary support for Command Console Files (CEL and CHP) note that not all applications support CC CEL and CHP files yet. See application specific notes below for which applications now support CC files.
Change to the PLIER algorithm
The PLIER source code contained in this release is replacing the separate stand alone PLIER SDK.
The PLIER code was changed by the addition of three new options:
SafetyZero: Within the PLIER code zero values are now adjusted by a slight positive bias (default 0.000001) to prevent errors during log transformation. This is particularly relevant when feature response values are rounded to zero and feed into PLIER as precomputed feature effects.
NumericalTolerance: Sets how hard the PLIER algorithm will work to resolve small differences between results when using precomputed vs non-precomputed feature responses. (Default is 0.1)
FixPrecomputed: This option causes the PLIER code to recompute the target response after fitting the feature response. This improves concordance between PLIER results generated de novo versus those generated using a precomputed feature response.
These changes improve the stability of PLIER estimates when comparing those generated using precomputed feature responses versus those generated using full PLIER model fit. Note that in this release of APT you may still see some significant differences in cases where the likelihood space is relatively flat combined with rounding errors caused by writing out floating point values to the text feature response file.
Change sketch normalization default sketch size
Sketch normalization (ie when you request quant-norm and do not specify a sketch size) will now default to a sketch size of the 1% of the features or 50,000 - which ever is higher. If there are fewer than 50,000 features, then all the features will be used (ie full quantile normalization).
New Applications:
apt-cel-extract: extract intensities from CEL files
apt-chp-to-txt: extract values from CHP files
apt-cel-to-pq: dump out mapping genotype probeset intensities as quartet
apt-dump-pgf: extract/dump out information in pgf and clf library files
apt-summary-vis: create IGB and BED visualization files from summary.txt files
apt-cdf-export: extract/dump out information in the cdf library files
apt-tsv-join: merge text files
apt-matrix-diff: compare two text files w/ matrix of values
Changed Applications:
apt-midas:
Changed -c option to --cel-files for consistency with other APT apps.
Changed header in file grouping CEL files from cel_file to cel_files for consistency with other APT apps.
apt-probeset-summarize:
meta probeset files are now specified using a separate option, -m or --meta-probesets rather than by using the -s option as before.
change --chp-output option to --xda-chp-output
change output/input files to use probe_id rather than feature_id for improved consistency with terminology used in various library files
apt-probeset-summarize will now try and guess an appropriate block size given the amount of available memory to reduce the change of out of memory errors and disk swap
support for Comand Console (CC) CHP file output and CEL file input
rm-probe option for masking out probes
transposed report file contents to be consistent with apt-probeset-genotype
reports associated with each quantification method such that the use of multiple analysis parameters (-a) will result in multiple reports
reports are always generated -- removed --report option
added an avgdiff and sea methods
added md5sum of probes used for GC background correction
added md5sum of probes used for computing quantile normalization sketch
refactored memory allocation to improve memory use on Windows where the address space is more fragmented
reduced memory used by the quantile normalization (including sketch) code
apt-probeset-genotype:
change --chp-output option to --xda-chp-output
rm-probe option for masking out probes
automatic blocking of probesets to improve usability on low memory machines
support for Comand Console (CC) CHP file output and CEL file input (CC genotype CHP file content is currently experimental and will change)
changed Unknown to UnknownGender
for xda CHP output, report median raw PM values for controls (in order to be consistent with CHP files generated by GTYPE)
the report file now had a column header for the cel file name, "cel_files"
Binary Releases
Added Mac OS-X Universal/Fat Binaries for i386/ppc/ppc64
Removed dependency on .NET for windows binaries
Source Code
draft file format documentation
refactor common code out of the application specific engines
refactor the QC reports to allow for probe and probeset level metrics
formalizing structure of options that are used by chipstream via SelfDoc interface. There is now a SelfDoc::Opt class which contains a name, description, default and range values for each available option.
include Command Console file parsing code
allow for serialization of engine parameters
clean up code to quell gcc warnings
enable building fat/universal binaries on OS-X
cancel engines is now done through the ProgressHandlers
refactor general enumeration and types not specific to any one class or method
Version:
apt-1.4.0
Date:
Wed Apr 26 2006
This is the first stand alone release of the Affymetrix Power Tools (APT). Previous versions were released as part of the Exon Array Computational Tool (ExACT).
General
The default APT license is now GPL. For commercial override contact devnet@affymetrix.com.
New Applications
apt-cel-transformer: create new CEL files with various data transformations applied (ie normalization)
apt-snp-compare: simple tool for comparing matrix of SNP calls
Changed Applications
apt-probeset-genotype: implements the BRLMM genotype calling algorithm for mapping arrays
apt-probeset-summarize: generate probeset signal estimates from expression and exon arrays
changed name to include apt prefix
change feature response and residual output (and input) files now have probeset_id, atom_id, feature_id (aka probe_id), x, and y values.
change default precision of output (0 places for most files, 5 digits for dabg p values)
added blocking -- this tells the software to only process a certain number of probesets at a time in order to keep the memory requirement down.
default behavior when using non PM-only adjuster w/ med-polish (RMA) is to now attenuate the mismatch using glog rather than simply capping values at a slightly positive value
no longer experimental
apt-midas: compute the MiDAS F stat and P values for alternative splicing
added apt prefix to name
Binary Releases
MS Windows installer
Prebuilt RedHat Linux Fedora Core 3 i386 32-bit static binaries
Prebuilt RedHat Linux Fedora Core 3 AMD 64-bit static binaries
Source Code
Broken out main probeset-genotype and probeset-summarize into a core engine and a separate thin command line wrapper.
Refactor status and log reporting to facilitate GUI integration
Migrate windows builds to Visual Studio 2005
New build system for *NIX systems
More doxygen high level and API documentation
Version:
apt-1.2.1 (as part of ExACT 1.2.1)
Date:
Mon Jan 30 2006
Minor changes on the C++ side to support re-applying a multi-chip PLIER analysis to new chips.
Changed Applications
probeset-summarize
added option to use pre-computed normalization sketches and precomputed feature responses
added option to save sketches and precomputed feature responses
WARNING, because of the way the PLIER algorithm alternates optimizing the signal estimate and the feature responses, using precomputed feature responses will result in a slightly different answer when the same CEL files are used.
Version:
apt-1.2.0 (as part of ExACT 1.2.0)
Date:
Mon Jan 02 2006
This was essentially the first release of APT. Most of the C++ code was experimental and thus subject to change. You may try the devnet@affymetrix.com mail address or the DevNet ExACT message forum on Affymetrix.com for unofficial support.
New Applications
midas: compute the MiDAS F stat and P values for alternative splicing, the MiDAS method is described in a whitepaper available here.
apt-probeset-summarize: generate probeset signal estimates from expression and exon arrays
does normalization and signal summarization in a single step
includes RMA, IterPLIER, and PLIER
experimental
Source Code
Added C++ DABG implementation. See the source code under sdk/dabg.
Added experimental C++ implementation of the MiDAS altsplice detection method. See the source code under sdk/midas.
Added experimental C++ ChipStream code. This code provides a framework to apply multiple transformations in a multichip analysis. See code under sdk/chipstream.
A command line program using ChipStream, probeset-summarize, is also provided. This program can do normalization and probe summarization (ie PLIER, RMA, ...) in a single go and can do so with either a CDF file or a PGF/CLF files. See code under sdk/chipstream/probeset-summarize.
Added a separate build system for the C++ code. The makefile will build a simple Affymetrix SDK static library and the chip stream (probeset-summarize) and midas executables. If you have CPP Unit installed, some basic unit testing will also be done.
Affymetrix Power Tools (APT) Release apt-1.10.1
Generated on Mon Nov 3 12:21:41 2008 for Affymetrix Power Tools by
1.5.3
