Open Access

On the necessity of different statistical treatment for Illumina BeadChip and Affymetrix GeneChip data and its significance for biological interpretation

Biology Direct20083:23

DOI: 10.1186/1745-6150-3-23

Received: 05 May 2008

Accepted: 03 June 2008

Published: 03 June 2008

Abstract

Background

The original spotted array technology with competitive hybridization of two experimental samples and measuring relative expression levels is increasingly displaced by more accurate platforms that allow determining absolute expression values for a single sample (for example, Affymetrix GeneChip and Illumina BeadChip). Unfortunately, cross-platform comparisons show a disappointingly low concordance between lists of regulated genes between the latter two platforms.

Results

Whereas expression values determined with a single Affymetrix GeneChip represent single measurements, the expression results obtained with Illumina BeadChip are essentially statistical means from several dozens of identical probes. In the case of multiple technical replicates, the data require, therefore, different stistical treatment depending on the platform. The key is the computation of the squared standard deviation within replicates in the case of the Illumina data as weighted mean of the square of the standard deviations of the individual experiments. With an Illumina spike experiment, we demonstrate dramatically improved significance of spiked genes over all relevant concentration ranges. The re-evaluation of two published Illumina datasets (membrane type-1 matrix metalloproteinase expression in mammary epithelial cells by Golubkov et al. Cancer Research (2006) 66, 10460; spermatogenesis in normal and teratozoospermic men, Platts et al. Human Molecular Genetics (2007) 16, 763) significantly identified more biologically relevant genes as transcriptionally regulated targets and, thus, additional biological pathways involved.

Conclusion

The results in this work show that it is important to process Illumina BeadChip data in a modified statistical procedure and to compute the standard deviation in experiments with technical replicates from the standard errors of individual BeadChips. This change leads also to an improved concordance with Affymetrix GeneChip results as the spermatogenesis dataset re-evaluation demonstrates.

Reviewers

This article was reviewed by I. King Jordan, Mark J. Dunning and Shamil Sunyaev.

Background

Microarrays that rely on hybridization with DNA probes pioneered large-scale expression studies. After the introduction of spotted array technology in the mid 90s, microarrays have steadily gained popularity for exploratory gene expression analysis. A spotted array experiment requires both the treated and control samples to be labeled with different dyes and to be competitively hybridized on the same array. The expression level is expressed as a ratio between the intensities between the two labels. Spotted arrays are plagued by accuracy and sensitivity problems that are only partly remedied by the measuring only relative expression. Dye bias and repeatability remain unsatisfactory.

In recent years, Affymetrix GeneChip and Illumina BeadChip have emerged as two of the most popular microarray platforms. From the experimental design viewpoint, the GeneChip and BeadChip offer flexibility in terms of their ability to measure absolute expression values for each experimental sample independently. The growing amount of publicly available microarray data has prompted researchers to explore ways to compare results between experiments across the different platforms. This task signifies the first step in producing consistent and trusted results to support meaningful biological discovery. Yet, it is more difficult than it appears superficially. Even a simple variant of the problem like comparing results from the same sample across different platforms is not trivial. The first step requires the statistically significant changes in gene expression to be determined between treatment conditions for each platform. The platform-specific gene lists generated by applying the same significance threshold are finally compared. In general, the concordance between these gene lists is disappointingly low. Nevertheless, recent works have shown that concordance improvements can be made by filtering for gene nucleotide sequence identity [14], by suppressing lower intensity genes [5] or by aligning gene lists with continuous measures of differential gene expression [6].

During our evaluation of cross-platform comparison between Affymetrix and Illumina, we stumbled upon another, quite surprising reason for the low concordance. Given the specific design of Illumina arrays [710], it appears that the data derived from them requires specific statistical treatment different from that of more classical microarrays. Notably, the Affymetrix GeneChip and Illumina BeadChip have one stark difference in their designs. In a nutshell, many instances of a unique probe design are synthesized onto a group of adjacent discrete features or cells on the GeneChip. Consequently, each group of cells will target a particular gene. In the case of Illumina BeadChip, a unit of bead coated with hundred of thousands of probes is analogous to a group of cells on GeneChip. Furthermore, multiple beads of a probe design are immobilized onto randomized positions on the BeadChip. Therefore, given a probe design, a gene is only measured once on the GeneChip, whereas it is measured typically about 30 times on the Beadchip. But instead of delivering the individual bead intensities (possible with appropriate scanner modifications), the mean and standard error (i.e., the standard deviation divided by the square root of the number of beads) of the bead intensities, known as the summary data, are usually reported.

Thus, Affymetrix GeneChips provide individual measurement results but the Illumina BeadChips generate means and standard errors for subsets of bead intensity measurements. Therefore, the summary data of a BeadChip experiment requires a different statistical interpretation compared with the individual measurements in the case of GeneChip data, especially in cases of multiple technical replicates. If the average of the bead intensities delivered by a single Illumina BeadChip is fed into standard expression profile analysis software (for example, GeneSpring), the standard deviation over technical replicates is calculated from the deviations of the subset means from the overall mean. But more correctly, the overall standard error is to be computed by taking into account also the standard deviations obtained from the individual BeadChips.

In this paper, we will first present a derivation for the correct summary statistic applicable to Illumina BeadChips data. Furthermore, this summary statistic will be applied to one control experiment with artificial spikes and, also, it will be used for the re-evaluation of two published biological experiments. In all cases, the modified treatment is contrasted against the standard one. In the control experiment [11], we will demonstrate a dramatic improvement in recognizing the spike sequence selection if the corrected summary statistic is applied. In the example of the MT1-MMP mammary epithelium dataset [7], cell cycle pathway involvement can be shown with statistical confidence only after applying the correct summary statistic. Interestingly, cell cycle gene involvement was suggested by the authors, although their analysis of the data did not provide strong arguments for it. Then with the spermatogenesis cross-platform data [8], we demonstrate that considerably improved concordance between the Affymetrix and Illumina platforms can be achieved with the correct summary statistic. Our analysis also provides new evidence for the transcriptional regulation of the N-glycan biosynthesis, the tight and the adherens junction pathways in this context, a finding that is supported also by independent experimental evidence.

Results & Discussion

Statistics of Illumina BeadChip & Affymetrix GeneChip datasets

The bead intensity of a given gene in a BeadChip is described with the random variable X. The expression profile experiment is supposed to consist of K technical replicates (independent measurement of arrays on the same biological sample). Each bead intensity xk,nis an instance of the random variable X (where k = 1...K replicates, n = 1...N k beads, N k is the number of beads in the k-th technical replicate). We assume that the first M k beads are retained after outlier removal (see below). The summary data includes the mean μ k , the standard error σ k / M k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdUgaRbqabaGccqGGVaWldaGcaaqaaiabd2eannaaBaaaleaacqWGRbWAaeqaaaqabaaaaa@32D1@ (where σ k is the standard deviation) and the number of beads M k (the typical value of M k is about 30).

The observed BeadChip intensities of gene in the k-th array are denoted as X k ¯ = [ x k , 1 x k , 2 x k , N k ] T MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaa0aaaeaacqWGybawdaWgaaWcbaGaem4AaSgabeaaaaGccqGH9aqpcqGGBbWwfaqabeqaeaaaaeaacqWG4baEdaWgaaWcbaGaem4AaSMaeiilaWIaeGymaedabeaaaOqaaiabdIha4naaBaaaleaacqWGRbWAcqGGSaalcqaIYaGmaeqaaaGcbaGaeS47IWeabaGaemiEaG3aaSbaaSqaaiabdUgaRjabcYcaSiabd6eaonaaBaaameaacqWGRbWAaeqaaaWcbeaaaaGccqGGDbqxdaahaaWcbeqaaiabdsfaubaaaaa@45FF@ (Table 1). However, a typical BeadChip experiment does not report these individual bead intensities. Instead, the Illumina BeadStudio software first performs an outlier removal on the bead intensities. Instances with intensities above three median absolute deviations from the median are removed. Upon the outlier removal, the mean and standard error of the bead intensities as well as the number of beads used in summarization for each gene are reported (the AVG_signal, BEAD_STDERR, Avg_NBEADS columns in the Illumina Beadstudio output file).
Table 1

Intensity output of Illumina & Affymetrix across K technical replicates

Platform

Replicate 1

Replicate 2

Replicate K

Illumina BeadChip (Raw data)

X 1 ¯ = [ x 1 , 1 x 1 , 2 x 1 , N 1 ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaa0aaaeaacqWGybawdaWgaaWcbaGaeGymaedabeaaaaGccqGH9aqpdaWadaqaauaabeqaeeaaaaqaaiabdIha4naaBaaaleaacqaIXaqmcqGGSaalcqaIXaqmaeqaaaGcbaGaemiEaG3aaSbaaSqaaiabigdaXiabcYcaSiabikdaYaqabaaakeaacqWIUlstaeaacqWG4baEdaWgaaWcbaGaeGymaeJaeiilaWIaemOta40aaSbaaWqaaiabigdaXaqabaaaleqaaaaaaOGaay5waiaaw2faaaaa@41E8@

X 2 ¯ = [ x 2 , 1 x 2 , 2 x 2 , N 2 ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaa0aaaeaacqWGybawdaWgaaWcbaGaeGOmaidabeaaaaGccqGH9aqpdaWadaqaauaabeqaeeaaaaqaaiabdIha4naaBaaaleaacqaIYaGmcqGGSaalcqaIXaqmaeqaaaGcbaGaemiEaG3aaSbaaSqaaiabikdaYiabcYcaSiabikdaYaqabaaakeaacqWIUlstaeaacqWG4baEdaWgaaWcbaGaeGOmaiJaeiilaWIaemOta40aaSbaaWqaaiabikdaYaqabaaaleqaaaaaaOGaay5waiaaw2faaaaa@41F2@

X K ¯ = [ x K , 1 x K , 2 x K , N K ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWaa0aaaeaacqWGybawdaWgaaWcbaGaem4saSeabeaaaaGccqGH9aqpdaWadaqaauaabeqaeeaaaaqaaiabdIha4naaBaaaleaacqWGlbWscqGGSaalcqaIXaqmaeqaaaGcbaGaemiEaG3aaSbaaSqaaiabdUealjabcYcaSiabikdaYaqabaaakeaacqWIUlstaeaacqWG4baEdaWgaaWcbaGaem4saSKaeiilaWIaemOta40aaSbaaWqaaiabdUealbqabaaaleqaaaaaaOGaay5waiaaw2faaaaa@42D3@

Illumina BeadChip (Summary data)

μ 1 , σ 1 M 1 , M 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabigdaXaqabaGccqGGSaaljuaGdaWcaaqaaiabeo8aZnaaBaaabaGaeGymaedabeaaaeaadaGcaaqaaiabd2eannaaBaaabaGaeGymaedabeaaaeqaaaaakiabcYcaSiabd2eannaaBaaaleaacqaIXaqmaeqaaaaa@3870@

μ 2 , σ 2 M 2 , M 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabikdaYaqabaGccqGGSaaljuaGdaWcaaqaaiabeo8aZnaaBaaabaGaeGOmaidabeaaaeaadaGcaaqaaiabd2eannaaBaaabaGaeGOmaidabeaaaeqaaaaakiabcYcaSiabd2eannaaBaaaleaacqaIYaGmaeqaaaaa@3878@

μ K , σ K M K , M K MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabdUealbqabaGccqGGSaaljuaGdaWcaaqaaiabeo8aZnaaBaaabaGaem4saSeabeaaaeaadaGcaaqaaiabd2eannaaBaaabaGaem4saSeabeaaaeqaaaaakiabcYcaSiabd2eannaaBaaaleaacqWGlbWsaeqaaaaa@392C@

Affymetrix GeneChip (Raw data)

x 1,1

x 2,1

X K,1

Using the means and standard errors of all the technical replicates, the mean μ total and standard deviation σ total of bead intensities of a gene across the K technical replicates are given as
μ t o t a l = 1 K k = 1 K μ k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabdUealbaakmaaqahabaGaeqiVd02aaSbaaSqaaiabdUgaRbqabaaabaGaem4AaSMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaaaa@42AD@
(1)
σ t o t a l = σ μ 2 + σ w t r e p 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpdaGcaaqaaiabeo8aZnaaDaaaleaacqaH8oqBaeaacqaIYaGmaaGccqGHRaWkcqaHdpWCdaqhaaWcbaGaem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabaGaeGOmaidaaaqabaaaaa@459E@
(2)
where  σ μ = 1 K k = 1 K μ k 2 [ 1 K k = 1 K μ k ] 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaeeiiaaIaeq4Wdm3aaSbaaSqaaiabeY7aTbqabaGccqGH9aqpdaGcaaqaaKqbaoaalaaabaGaeGymaedabaGaem4saSeaaOWaaabCaeaacqaH8oqBdaqhaaWcbaGaem4AaSgabaGaeGOmaidaaaqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOGaeyOeI0YaamWaaeaajuaGdaWcaaqaaiabigdaXaqaaiabdUealbaakmaaqahabaGaeqiVd02aaSbaaSqaaiabdUgaRbqabaaabaGaem4AaSMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaakiaawUfacaGLDbaadaahaaWcbeqaaiabikdaYaaaaeqaaaaa@56D3@
(3)
σ w t r e p = M 1 σ 1 2 + M 2 σ 2 2 + ... + M K σ K 2 M 1 + M 2 + ... + M K MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaGccqGH9aqpdaGcaaqcfayaamaalaaabaGaemyta00aaSbaaeaacqaIXaqmaeqaaiabeo8aZnaaDaaabaGaeGymaedabaGaeGOmaidaaiabgUcaRiabd2eannaaBaaabaGaeGOmaidabeaacqaHdpWCdaqhaaqaaiabikdaYaqaaiabikdaYaaacqGHRaWkcqGGUaGlcqGGUaGlcqGGUaGlcqGHRaWkcqWGnbqtdaWgaaqaaiabdUealbqabaGaeq4Wdm3aa0baaeaacqWGlbWsaeaacqaIYaGmaaaabaGaemyta00aaSbaaeaacqaIXaqmaeqaaiabgUcaRiabd2eannaaBaaabaGaeGOmaidabeaacqGHRaWkcqGGUaGlcqGGUaGlcqGGUaGlcqGHRaWkcqWGnbqtdaWgaaqaaiabdUealbqabaaaaaWcbeaaaaa@5AB0@
(4)
A proof for equation (2) is supplied in the Appendix 1. The standard deviation σ total is composed of two components each carrying a different meaning. Given that each of the K technical replicates represents the same, identical and independent distribution, one expects the K mean estimates μ k to be relatively similar and, hence, σ μ would be small relative to σ total and, ideally, close to zero. Since averaging for each replicate is carried out over about 30 individual measurements, it can be assumed that each individual μ k is likely a good estimate of the population mean μ if there were no batch variations. Therefore, a large σ μ can be interpreted as batch variation or noise among the replicates, a considerable part of which, apparently, has systematic origin such as variations in the total amount of hybridization-ready nucleic acids, etc. Ideally, K (typically 2–4) should be much larger in order to obtain a good estimate of σ μ . However, this is impractical due to the high cost of performing large number of microarrays. Therefore, we suggest to assume σ μ ≈ 0 for the case of no batch variation in equation (5) and to use σ wtrep as a good lower estimate of the summary statistic in testing instead of σ μ (see Appendix 2 for proof).
σ t o t a l = σ w t r e p 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpdaGcaaqaaiabeo8aZnaaDaaaleaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeaacqaIYaGmaaaabeaaaaa@401A@
(5)

This proposed summary statistic is supported by observations communicated in two recent publications, which have leveraged on the variation in bead intensities. Dunning et al. [12] showed that differentially expressed gene detection experienced an increase in statistical power by using the inverse of σ k 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdUgaRbqaaiabikdaYaaaaaa@3016@ as weights in their linear model. On the other hand, Lin et al. [13] proposed a variance stabilization transformation that incorporated bead intensities variation and showed an improvement in differentially expressed gene detection. Beyond this point, we shall refer to σ total with respect to equation (5) instead of (2).

In the case of a Affymetrix GeneChip experiment, the measurement for a gene is taken only once in each replicate (see 4th row of Table 1). Consequently, the mean and standard deviation of a gene across K technical replicates are given as
ν t o t a l = 1 K k = 1 K x k , 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqyVd42aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabdUealbaakmaaqahabaGaemiEaG3aaSbaaSqaaiabdUgaRjabcYcaSiabigdaXaqabaaabaGaem4AaSMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaaaa@4442@
(6)
ω t o t a l = 1 K k = 1 K ( x k , 1 v t o t a l ) 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqyYdC3aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpdaGcaaqaaKqbaoaalaaabaGaeGymaedabaGaem4saSeaaOWaaabCaeaacqGGOaakcqWG4baEdaWgaaWcbaGaem4AaSMaeiilaWIaeGymaedabeaakiabgkHiTiabdAha2naaBaaaleaacqWG0baDcqWGVbWBcqWG0baDcqWGHbqycqWGSbaBaeqaaOGaeiykaKYaaWbaaSqabeaacqaIYaGmaaaabaGaem4AaSMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaaleqaaaaa@50DA@
(7)

The summary statistic [μ total , σ total ] and [ν total , ω total ] are the parallels between the Illumina and Affymetrix platforms. However, σ total has an advantage over ω total . Due to multiple copies of the same probe within a single Illumina array, the standard deviation can be computed for each array individually. As a result, σ total offers more protection against any systematic error than ω total (see Appendix 2 for proof). The lack of systematic error as a confounding factor in σ total increases the chance of detecting true biological differences from the statistical tests.

In any case, the more important concern related to the analysis of Illumina data is the mistake of treating the mean estimates of bead intensities as instances of the bead intensities. Standard gene expression profile analysis software (as applied in several published studies [710]) assumes that the imported data are bead intensities rather than mean estimates of bead intensities. Such a software plainly computes the mean and standard deviation for the incoming data and the corresponding summary statistic for the control and the treatment group would be [μ total , σ μ ] control and [μ total , σ μ ] treatment respectively. The summary statistic σ μ is incorrect since it measures only the batch variation and not at all variation in bead intensities. The correct summary statistic should be [μ total , σ wtrep ] control and [μ total , σ wtrep ] treatment . The emphasis of using σ wtrep instead of σ μ as the summary statistic is not statistical hair splitting but this issue affects the biological interpretation as we can see from the following three examples.

Illumina spike data: improvement in p value ranking

Chudin et al. [14] provided Illumina BeadChip data for 34 spikes (varying concentrations from 0.01 to 1000 pM) against a background of human cRNA. To examine the effects of using σ μ instead of σ wtrep as summary statistic, each pair of spike experiments at adjacent concentrations were compared (Table 2). In each array, there are 34 spike and 48730 non-spike transcript probes. This is equivalent to 34 true alternative hypotheses and 48730 true null hypotheses. Ideally, the P-values of the 34 alternate hypotheses will appear on one extremity of the Schweder-Spojotvoll plot [15]. Upon computing and sorting the P-values of the pair-wise t-tests (after array normalization as in Materials and methods), each of the 34 smallest P-values was examined to see if it belongs to a true positive (TP) or false positive (FP) gene.
Table 2

Number of TP and FP genes based on P-value ranking.

 

σ μ

σ wtrep

 

Concentration (in pM)

TP

FP

TP

FP

No. of common TP

0.01 vs 0

0

34

0

34

0

0.03 vs 0.01

0

34

0

34

0

0.1 vs 0.03

7

27

9

25

7

0.3 vs 0.1

14

20

22

12

14

1 vs 0.3

30

4

33

1

30

3 vs 1

30

4

34

0

30

10 vs 3

33

1

34

0

33

30 vs 10

34

0

34

0

34

100 vs 30

33

1

34

0

33

300 vs 100

4

30

26

8

4

1000 vs 300

9

25

16

18

8

The first column indicates the 11 test cases of pair-wise treatment conditions. The second and third columns indicate the number of TP and FP genes respectively for using σ μ as the summary statistic. Similarly, column four and five indicate the results for using σ wtrep . The last column indicates the result for the number of common TP genes found by both σ μ and σ wtrep .

The number of identified TP genes by the statistic σ wtrep is generally higher than that by σ μ (Table 2). In particular, an improvement from 7 (0.1 and 0.03 pM comparison) or 14 (0.3 versus 0.1 pM) to 9 and 22 recovered spikes in the low concentration range of 0.03–0.3 pM is encouraging. Note that this region spans the endogenous gene expression level and, hence, it is critical to obtain good differentially expressed gene identification here. An improvement was also achieved in the high concentration region. But in practice, gene expression will not reach such level to leverage on it. Note that the detection limit was 0.25 pM while the saturation point was about 300 pM [11].

Most importantly, the TP genes found by σ μ is a subset of those found by σ wtrep . This means that more TP genes found by σ wtrep had moved into the first 34 ranks to displace only other FP genes. For that to happen, the P-values must have been re-ranked by the statistic so that the TP genes are more statistically significant than the FP genes.

This means that σ μ is not a good estimate for the standard deviation. Figure 1 shows a plot of mean bead intensities μ total against the standard deviation of mean estimates σ μ and variation in bead intensities σ wtrep of the human background cRNA (i.e. 0 pM spike data). 'Heteroskedasticity' means that the larger intensities tend to have larger variations, a common observation with many types of microarray data. The 'heteroskedasticity' nature of the relationship between the mean bead intensities μ total and the variation in bead intensities σ wtrep is apparent (in red in Figure 1). On the other hand, a trend of growth in the standard deviation of mean estimates σ μ with increase of mean intensities across the dynamic range is not certain (in blue in Figure 1) and even σ μ = const cannot be excluded (the case of purely systematic error). Given that the number of technical replicates K is only four, obtaining good estimates for σ μ especially in the higher intensities region is impossible. As such, we strongly advocate the use of equation (5), which only relies on σ wtrep for computing the correct summary statistic, instead of equation (2).
https://static-content.springer.com/image/art%3A10.1186%2F1745-6150-3-23/MediaObjects/13062_2008_Article_101_Fig1_HTML.jpg
Figure 1

Relationship between σ μ , σ wtrep and μ total . We show the plot of mean estimates of bead intensities μ total against standard deviation in mean estimates σ μ and bead intensities σ wtrep of 0 pM spike concentration data.

MT1-MMP data: proof for cell cycle pathway involvement

Golubkov et al. [7] published the expression profiles of mammary epithelial cells without and after transfection with a plasmid carrying the membrane type-1 matrix metalloproteinase (MT1-MMT) gene recorded with the Illumina platform. Originally, the expression data was first normalized using the "normalize.quantiles" [16] routine of Bioconductor and then imported into GeneSpring for Welch's t-test (thus, using σ μ as the summary statistic). A total of 207 differentially expressed genes were determined with cutoff criteria of p ≤ 0.05 and absolute fold change (FC) of at least 2.

In this work, the original expression data was first normalized (see Array normalization procedure section) prior to statistical treatment. Welch's t test was then performed for both σ μ and σ wtrep , which yielded 215 and 218 differentially expressed genes respectively upon applying the same cutoff criteria. For the three lists consisting of 207, 215 and 218 gene candidates, RefSeq IDs were extracted. The resulting 202, 200 and 203 RefSeq IDs were then separately submitted to NIH DAVID [17] for KEGG pathway mapping. Furthermore, 19815 RefSeq IDs were extracted from the Illumina Human-6 Expression BeadChip annotation file and submitted to DAVID as the background list.

With reference to Table 3, only KEGG pathways with EASE[18] score ≤ 0.05 and count ≥ 5 are shown. In a nutshell, the EASE score is a P- value of a more conservative version of the Fisher's exact test while count denotes the number of genes in the differentially expressed gene list that belongs to a particular pathway. Notably, our analysis with the statistic σ μ essentially repeats the outcome of the work by Golubkov et al. with respect to the pathways (Table 3) and regulated genes (the overlap between the two lists includes 176 genes out of 200 and 202 genes respectively); thus, the differences in the normalization had a small effect.
Table 3

KEGG pathways elucidated from the MT1-MMP data.

KEGG pathway

σ μ

[7]

σ μ

σ wtrep

HSA01430 : Cell communication

HSA04540 : Gap junction

HSA04610 : Complement and coagulation cascades

HSA04110 : Cell cycle

  

The first column indicates the name of the KEGG pathways. Only KEGG pathways with EASE score ≤ 0.05 and count ≥ 5 are included. The second to fourth columns indicate KEGG pathway found by Golubkov V.S. et. al. [7], σ μ and σ wtrep respectively. Cutoff criteria of EASE score ≤ 0.05 and count ≥ 5 was applied.

Application of the statistic σ wtrep dramatically influences the result. Suddenly, additional cell cycle genes are significantly regulated in transfected cells and the cell cycle pathway pops up in the DAVID analysis. Table 4 highlights the two genes out of a total of six significantly up-regulated genes from the elucidated cell cycle pathway. These two genes, Cyclin A1 and CDC45L, are found by σ wtrep but not by σ μ . Consequently, the addition of these two genes resulted in an improved EASE score of 0.04 (from 0.29). The elucidation of this pathway has substantiated the authors' claim with statistically significant expression arguments that the cell cycle is disrupted with observable mitotic spindle aberrations and aneuploidy in the 184B5-MT cells [7].
Table 4

Cell cycle genes in the MT1-MMP data.

  

σ μ [7]

σ wtrep

 

Gene Symbol

RefSeq ID

log2FC

p value

log2FC

p value

Gene Description

CCNA1

NM_003914

-

≥ 0.05

1.12

0.00

Cyclin A1

CDC45L

NM_003504

-

≥ 0.05

1.05

0.00

CDC45 cell division cycle 45-like

CCNB1

NM_031966

1.30

< 0.05

1.44

0.00

Cyclin B1

CCNB2

NM_004701

1.27

< 0.05

1.39

0.00

Cyclin B2

CDC2

NM_033379

1.10

< 0.05

1.20

0.00

cell division cycle 2, G1 to S and G2 to M

CDC20

NM_001255

2.02

< 0.05

2.01

0.00

Cell division cycle 20 homolog

The first, second and last column indicate the gene symbol, the RefSeq ID and the description of each cell cycle gene respectively. The third and fourth columns indicate the logarithm fold change and the P-value of each gene found by Golubkov V.S. et. al. [7], which is analogous to using σ μ as the summary statistic. The fifth and sixth columns indicate the logarithm fold change and the P-value of each gene found by using σ wtrep as the summary statistic. Cutoff criteria of p ≤ 0.05 and |FC| ≥ 2 were applied. The genes in the upper section were found only by the σ wtrep statistic, whereas the genes in the lower section were detected with significance by both statistics.

Human spermatogenesis data: proof for the N-glycan, the tight and the adherens junction pathway involvement

Platts et al. [8] studied RNA expression in ejaculates of normal and zoospermic men both with the Affymetrix and the Illumina platforms. The Affymetrix expression data of 13 normal and 8 teratozoospermic men was processed by the MBEI (PM-MM) algorithm after invariant set normalization to obtain the gene expression values using the DChip software [19]. The Illumina BeadChip study included only 5 out of the 13 normal but all zoospermic examples. The authors used the same procedure for elucidating differentially expressed genes in both cases [8].

In this work, 5 out of the 13 normal and the 8 teratozoospermic samples from the Affymetrix experiment that were used by Platts et al. in their Illumina experiment (N1, N5, N6, N11, N12) were re-analyzed. The gene-level data was normalized (see Materials and methods), followed by a pair-wise t-test with ν and ω as the summary statistic (equations 6 and 7). This resulted in a total of 11932 differentially expressed genes (6861 RefSeq IDs) after applying cutoff criteria of p ≤ 0.01 and |FC| ≥ 2. In a similar fashion, the expression data from the corresponding 5 normal and 8 teratozoospermic of the Illumina experiment was normalized and statistically treated for both σ μ and σ wtrep . Using the same cutoff criteria, the two analyses yielded 2464 DEGs (2109 RefSeq IDs) and 4149 DEGs (3316 RefSeq IDs) respectively. Since the number of differentially expressed genes for σ wtrep is increased for the same cutoff criteria, this statistic exhibited a higher statistical power.

The three RefSeq ID lists were submitted to DAVID for KEGG pathway mapping. For Affymetrix, the background list was set to a list of 39647 ReqSeq IDs that was extracted from the HG-U133 (version 2) annotation file. For Illumina, the same list of 19815 RefSeq IDs from MT1-MMP example (see previous section) was submitted as the background.

Analogous to Table 3, only KEGG pathways with EASE score ≤ 0.05 and count ≥ 5 are shown in Table 5. The elucidation of the proteosome and ubiquitin mediated proteolysis pathways by the re-analyzed Affymetrix expression data is consistent with the authors' finding that there is a severe suppression of the proteosomal RNAs associated with the ubiquitin-proteasomal pathway (UPP) in the teratozoospermic samples. On the other hand, the Illumina analysis revealed the proteosome but not the ubiquitin mediated proteolysis pathway. Even though the count for the ubiquitin-mediated proteolysis pathway had increased from 11 to 15 when σ wtrep was used instead of σ μ , this increase only improved the EASE score from 0.07 to 0.066. This marginally missed the significance cutoff of ≤ 0.05. But more interestingly, the Illumina analysis was able to elucidate a few pathways involved in spermatogenesis [20], like the N-glycan biosynthesis [2123], adherens and tight junction [2426] when σ wtrep was used as the summary statistic.
Table 5

KEGG pathways elucidated from the human spermatogenesis data.

KEGG pathway

Affymetrix

Illumina

 

ω

σ wtrep

σ μ

HSA00190 : Oxidative phosphorylation

HSA00970 : Aminoacyl-tRNA synthetases

HSA03010 : Ribosome

HSA03050 : Proteosome

HSA00010 : Glycolysis/Gluconeogenesis

 

HSA00030 : Pentose phosphate pathway

 

HSA00193 : ATP synthesis

 

HSA00530 : Aminosugars metabolism

 

HSA00640 : Propanoate metabolism

 

HSA03020 : RNA polymerase

 

HSA03060 : Protein export

 

HSA04110 : Cell cycle

 

MMU03010 : Ribosome

 

HSA04120 : Ubiquitin mediated proteolysis

  

HSA00020 : Citrate cycle (TCA cycle)

  

HSA00240 : Pyrimidine metabolism

 

 

HSA00251 : Glutamate metabolism

 

 

HSA00510 : N-glycan biosynthesis

 

 

HSA03022 : Basal transcription factors

 

 

HSA04520 : Adherens junction

 

 

HSA04530 : Tight junction

 

 

HSA00620 : Pyruvate metabolism

  

The first column indicates the name of the KEGG pathways. Only KEGG pathways with EASE score ≤ 0.05 and count ≥ 5 are included. The second column indicates KEGG pathways found by Affymetrix. The third and fourth columns indicate the KEGG pathways found by using σ μ and σ wtrep respectively. Cutoff criteria of EASE score ≤ 0.05 and count ≥ 5 were applied.

Table 6 highlights 8 out of 17 genes from the elucidated N-glycan pathway that are found by σ wtrep but not σ μ . Using σ wtrep as the summary statistic, the P-values for these 8 genes were improved. With the addition of these 8 genes, the EASE score improved from 0.143 to 0.003. More notably, genes with biological evidence on the role of N-glycan biosynthesis pathway in spermatogenesis begin to surface with the application of the correct summary statistic. There is independent experimental evidence that proves the involvement of the detected genes. For example, beta-1,4-galactosyltransferase-I (B4GALT1) is found to bind with ZP3 receptors on the sperm surface [27]. Also, there is a reported increase in dehydrodolichyl diphosphate synthase (DHDDS) activity in prepuberal rats during early stages of spermatogenesis [28, 29]. MAN2A2 is also found to be implicated in male infertility of the alpha-mannosidase IIx (MX) gene knockout mouse [2123].
Table 6

N-glycan biosynthesis genes in human spermatogenesis data.

  

σ μ

σ wtrep

 

Gene Symbol

RefSeq ID

log2FC

p value

log2FC

p value

Gene Description

B4GALT1

NM_001497

-0.45

0.499

-1.08

0.000

UDP-Gal:betaGlcNAc beta 1,4- galactosyltransferase, polypeptide 1

DDOST

NM_005216

-2.22

0.067

-1.18

0.000

dolichyl-diphosphooligosaccharide-protein glycosyltransferase

DHDDS

NM_024887

3.60

0.049

3.62

0.000

dehydrodolichyl diphosphate synthase

DPM1

NM_003859

-0.65

0.320

-1.06

0.000

dolichyl-phosphate mannosyltransferase polypeptide 1, catalytic subunit

GANAB

NM_198334

1.45

0.045

2.12

0.000

glucosidase, alpha; neutral AB

MAN1A2

NM_006699

-1.27

0.019

-1.28

0.000

mannosidase, alpha, class 1A, member 2

MAN2A2

NM_006122

-0.81

0.001

-1.06

0.000

mannosidase, alpha, class 2A, member 2

UGCGL2

NM_020121

-0.97

0.003

-1.25

0.000

UDP-glucose ceramide glucosyltransferase-like 2

ALG2

NM_033087

-2.18

0.000

-2.20

0.000

asparagine-linked glycosylation 2 homolog (S. cerevisiae, alpha-1,3-mannosyltransferase)

ALG5

NM_013338

-1.90

0.000

-1.71

0.000

asparagine-linked glycosylation 5 homolog (S. cerevisiae, dolichyl-phosphate beta-glucosyltransferase)

ALG8

NM_024079

-1.36

0.001

-1.52

0.000

asparagine-linked glycosylation 8 homolog (S. cerevisiae, alpha-1,3-glucosyltransferase)

B4GALT2

NM_003780

2.33

0.006

2.29

0.000

UDP-Gal:betaGlcNAc beta 1,4- galactosyltransferase, polypeptide 2

MAN2A1

NM_002372

-1.21

0.055

-1.33

0.000

mannosidase, alpha, class 2A, member 1

MGAT4A

NM_012214

-1.84

0.001

-1.72

0.000

mannosyl (alpha-1,3-)-glycoprotein beta-1,4-N-acetylglucosaminyltransferase, isozyme A

OGT

NM_181673

-2.31

0.009

-2.12

0.000

O-linked N-acetylglucosamine (GlcNAc) transferase (UDP-N-acetylglucosamine:polypeptide-N-acetylglucosaminyl transferase)

RPN1

NM_002950

-1.46

0.001

-1.51

0.000

ribophorin I

RPN2

NM_002951

-1.95

0.000

-2.00

0.000

ribophorin II

The first, second and last column indicate the gene symbol, the RefSeq ID and the description of each N-glycan biosynthesis gene respectively. The third and fourth columns indicate the logarithm fold change and the P-value of each gene found by using σ μ as the summary statistic. The fifth and sixth columns indicate the logarithm fold change and the P-value of each gene found by using σ wtrep as the summary statistic. The cutoff criteria p ≤ 0.01 and |FC| ≥ 2 were applied. The first 8 genes are found to be statistically significant by σ wtrep only, the rest was found with both statistics.

In the case of the tight junction pathway, 17 out of 37 genes from this elucidated pathway are found by σ wtrep but not σ μ (Table 7). The improvement in the P-values of these additional 17 genes effected a marked improvement in EASE score from 0.206 to 0.012. For the adherens junction pathway, 14 out of 26 genes are found only by σ wtrep (Table 8). Consequently, the EASE score improved dramatically from 0.527 to 0.037. As a result of applying the correct summary statistic, the genes CLDN1, CSNK2A2, CTNNA1, JAM3, and TJP1 have surfaced from the analysis. Claudin-1 (CLDN1) is involved in the developmental regulation of the tight junctions in mouse testis [30], while casein kinase 2, alpha prime polypeptide (CSNK2A2) null male mice are infertile and their cells from spermatogonia to early spermatids suffer nuclear envelope protrusions, outer membrane swelling and inner membrane disruption [31]. Also, the assembly of adherens junctions between Sertoli and germ cells was associated with a transient induction in the steady-state mRNA and protein levels of cadherins and catenins. In particular, alpha-catenin (CTNNA1) expression was seen in semi-quantitative reverse transcription polymerase chain reaction and immunoblotting [32]. Furthermore, the expression of zona occludens 1 (TJP1) in rats is regulated in vitro during the assembly of inter-Sertoli tight junctions during spermatogenesis [33] while JAM-3 is a protein required for spermatid differentiation [34].
Table 7

Tight junction genes in human spermatogenesis data.

  

σ μ

σ wtrep

 

Gene Symbol

RefSeq ID

log2FC

P-value

log2FC

P-value

Gene Description

ACTG1

NM_001614

-0.81

0.064

-1.14

0.000

actin, gamma 1

CLDN1

NM_021101

-2.47

0.059

-1.63

0.000

claudin 1

CLDN16

NM_006580

-1.20

0.019

-1.55

0.000

claudin 16

CLDN5

NM_003277

1.15

0.286

1.42

0.000

claudin 5 (transmembrane protein deleted in velocardiofacial syndrome)

CLDN6

NM_021195

2.48

0.135

1.88

0.000

claudin 6

CSNK2A2

NM_001896

-1.00

0.014

-1.57

0.000

casein kinase 2, alpha prime polypeptide

CTNNA1

NM_001903

-0.93

0.009

-1.04

0.000

catenin (cadherin-associated protein), alpha 1, 102 kDa

EXOC3

NM_007277

-1.26

0.019

-1.43

0.000

exocyst complex component 3

EXOC4

NM_021807

-0.78

0.012

-1.04

0.000

exocyst complex component 4

GNAI2

NM_002070

1.02

0.110

1.52

0.000

guanine nucleotide binding protein (G protein), alpha inhibiting activity polypeptide 2

JAM3

NM_032801

-0.86

0.005

-1.07

0.000

junctional adhesion molecule 3

KRAS

NM_004985

-2.85

0.032

-2.43

0.000

v-Ki-ras2 Kirsten rat sarcoma viral oncogene homolog

MYH9

NM_002473

1.09

0.183

1.02

0.000

myosin, heavy chain 9, non-muscle

PPP2R2B

NM_181676

-1.22

0.020

-1.97

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit B, beta isoform

PPP2R3A

NM_002718

1.42

0.013

1.50

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit B", alpha

RAB13

NM_002870

-1.17

0.118

-1.68

0.000

RAB13, member RAS oncogene family

TJP1

NM_175610

-2.63

0.038

-2.10

0.000

tight junction protein 1 (zona occludens 1)

AKT3

NM_181690

-1.16

0.001

-1.16

0.000

v-akt murine thymoma viral oncogene homolog 3 (protein kinase B, gamma)

CDC42

NM_044472

-1.67

0.001

-1.51

0.000

cell division cycle 42 (GTP binding protein, 25 kDa)

CLDN11

NM_005602

-2.03

0.000

-2.06

0.000

claudin 11 (oligodendrocyte transmembrane protein)

CLDN14

NM_012130

2.92

0.006

2.90

0.000

claudin 14

CSDA

NM_003651

-2.28

0.000

-2.48

0.000

cold shock domain protein A

CSNK2B

NM_001320

-2.61

0.006

-2.69

0.000

casein kinase 2, beta polypeptide

CTNNA2

NM_004389

-1.64

0.002

-1.88

0.000

catenin (cadherin-associated protein), alpha 2

CTTN

NM_138565

-1.91

0.008

-2.05

0.000

cortactin

EPB41L3

NM_012307

-1.85

0.001

-1.71

0.000

erythrocyte membrane protein band 4.1-like 3

MYH10

NM_005964

-1.34

0.000

-1.35

0.000

myosin, heavy chain 10, non-muscle

MYL6

NM_079423

-1.62

0.000

-1.78

0.000

myosin, light chain 6, alkali, smooth muscle and non-muscle

PPP2CA

NM_002715

-2.69

0.002

-2.40

0.000

protein phosphatase 2 (formerly 2A), catalytic subunit, alpha isoform

PPP2CB

NM_004156

-1.43

0.005

-1.83

0.000

protein phosphatase 2 (formerly 2A), catalytic subunit, beta isoform

PPP2R1B

NM_181699

-2.14

0.003

-2.34

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit A, beta isoform

PPP2R1B

NM_181699

-1.47

0.011

-1.18

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit A, beta isoform

PPP2R2A

NM_002717

-1.53

0.002

-1.86

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit B, alpha isoform

PPP2R2B

NM_181677

2.51

0.007

2.29

0.000

protein phosphatase 2 (formerly 2A), regulatory subunit B, beta isoform

PRKCH

NM_006255

-1.20

0.002

-1.03

0.000

protein kinase C, eta

PTEN

NM_000314

-2.30

0.004

-1.61

0.000

phosphatase and tensin homolog (mutated in multiple advanced cancers 1)

RHOA

NM_001664

-1.81

0.001

-1.58

0.000

ras homolog gene family, member A

CTNNA3

NM_013266

-1.03

0.007

-0.92

0.000

catenin (cadherin-associated protein), alpha 3

The first, second and last column indicate the gene symbol, the RefSeq ID and the description of each Tight junction gene respectively. The third and fourth columns indicate the logarithm fold change and the P-value of each gene found by using σ μ as the summary statistic. The fifth and sixth columns indicate the logarithm fold change and the P-value of each gene found by using σ wtrep as the summary statistic. A cutoff criteria of p ≤ 0.01 and |FC| ≥ 2 was applied. The first 17 genes are found to be statistically significant by σ wtrep only. The last row contain a gene excluded by σ wtrep due to |FC| ≤ 2 although p ≤ 0.01. The genes upper in the section were found only by the σ wtrep statistic, the remainder was found with both approaches except of the last gene entry recovered only by the σ μ statistic (it was filtered out due the fold change criterion).

Table 8

Adherens junction genes in human spermatogenesis data.

  

σ μ

σ wtrep

 

Gene Symbol

RefSeq ID

log2FC

P-value

log2FC

P-value

Gene Description

ACTG1

NM_001614

-0.81

0.064

-1.14

0.000

actin, gamma 1

BAIAP2

NM_006340

1.32

0.016

1.27

0.000

BAI1-associated protein 2

CREBBP

NM_004380

-0.98

0.002

-1.11

0.000

CREB binding protein (Rubinstein-Taybi syndrome)

CSNK2A2

NM_001896

-1.00

0.014

-1.57

0.000

Casein kinase 2, alpha prime polypeptide

CTNNA1

NM_001903

-0.93

0.009

-1.04

0.000

catenin (cadherin-associated protein), alpha 1, 102 kDa

FER

NM_005246

-1.37

0.015

-1.73

0.000

fer (fps/fes related) tyrosine kinase (phosphoprotein NCP94)

IQGAP1

NM_003870

-2.66

0.026

-2.27

0.000

IQ motif containing GTPase activating protein 1

MAPK1

NM_002745

-1.56

0.014

-1.34

0.000

mitogen-activated protein kinase 1

MAPK3

NM_002746

1.75

0.042

1.88

0.000

mitogen-activated protein kinase 3

PTPRF

NM_130440

-0.91

0.000

-1.10

0.000

protein tyrosine phosphatase, receptor type, F

SMAD2

NM_005901

-1.27

0.027

-1.59

0.000

SMAD family member 2

TCF7

NM_003202

3.13

0.045

2.90

0.000

transcription factor 7 (T-cell specific, HMG-box)

TJP1

NM_175610

-2.63

0.038

-2.10

0.000

tight junction protein 1 (zona occludens 1)

WASL

NM_003941

-1.99

0.047

-1.42

0.000

Wiskott-Aldrich syndrome-like

ACP1

NM_004300

-1.82

0.009

-1.66

0.000

acid phosphatase 1, soluble

ACP1

NM_007099

-1.77

0.000

-1.70

0.000

acid phosphatase 1, soluble

CDC42

NM_044472

-1.67

0.001

-1.51

0.000

cell division cycle 42 (GTP binding protein, 25 kDa)

CSNK2B

NM_001320

-2.61

0.006

-2.69

0.000

casein kinase 2, beta polypeptide

CTNNA2

NM_004389

-1.64

0.002

-1.88

0.000

catenin (cadherin-associated protein), alpha 2

MAP3K7

NM_145333

-1.52

0.000

-1.39

0.000

mitogen-activated protein kinase kinase kinase 7

MAPK1

NM_138957

-1.19

0.005

-1.23

0.000

mitogen-activated protein kinase 1

MAPK1

NM_138957

-1.68

0.002

-2.03

0.000

mitogen-activated protein kinase 1

RHOA

NM_001664

-1.81

0.001

-1.58

0.000

ras homolog gene family, member A

SMAD4

NM_005359

-1.02

0.000

-1.07

0.000

SMAD family member 4

SORBS1

NM_015385

1.86

0.001

1.99

0.000

sorbin and SH3 domain containing 1

WASF3

NM_006646

-1.62

0.002

-1.80

0.000

WAS protein family, member 3

CTNNA3

NM_013266

-1.03

0.007

-0.92

0.000

catenin (cadherin-associated protein), alpha 3

The first, second and last column indicate the gene symbol, the RefSeq ID and the description of each Tight junction gene respectively. The third and fourth columns indicate the logarithm fold change and the P-value of each gene found by using σ μ as the summary statistic. The fifth and sixth columns indicate the logarithm fold change and the P-value of each gene found by using σ wtrep as the summary statistic. Cutoff criteria of p ≤ 0.01 and |FC| ≥ 2 were applied. The first 14 genes are found to be statistically significant by σ wtrep only. The lower part of the table lists genes found by both statistics except for the last row that contains a gene excluded by σ wtrep due to |FC| ≤ 2 (although p ≤ 0.01).

The concordance between the three RefSeq lists was next investigated. The results are shown in Figure 2. Through the derivation of a parallel summary statistic to the Affymetrix one, an addition of 423 concordance RefSeq IDs was recovered. This is an increase of 45% (423 out of 942) in the concordance region. The correct summary statistic also improved gene discovery. Out of the 916 RefSeq IDs that were unique to the analysis by σ wtrep , 8 (B4GALT1, DHDDS, MAN2A2, CLDN1, CSNK2A2, CTNNA1, JAM3, TJP1) were validated spermatogenesis genes from the N-glycan biosynthesis, tight and adherens junction pathways. These pathways were not reported by Affymetrix. Another interesting observation is that the RefSeq list obtained with σ μ as a statistic almost formed a subset of the list yielded by σ wtrep . In total, 93.7% of its ReqSeq IDs were found within the list generated with σ wtrep . At first glance, it seems that the effect of using σ μ rather than σ wtrep does not seem detrimental. However, the rankings of the P-values in this overlapped region were not preserved, similar to the case of our Illumina spike experiments analysis. As such, the top 100 candidate list for example will be quite different when σ μ instead of σ wtrep is used.
https://static-content.springer.com/image/art%3A10.1186%2F1745-6150-3-23/MediaObjects/13062_2008_Article_101_Fig2_HTML.jpg
Figure 2

Venn diagram of gene list overlap. The Venn diagram of the distribution of differentially expressed genes (based on the cutoff criteria p ≤ 0.01 and |FC| ≥ 2) between Affymetrix and Illumina spermatogenesis datasets is presented.

Conclusion

Due to the specific statistical nature of the Illumina BeadChip summary data as means and standard deviations of subsets of measurements, the typical statistical workflow of finding differentially expressed genes cannot be applied to this data directly. To remedy this situation, σ wtrep is proposed as correct summary statistic of the Illumina BeadChip. Our work has shown that the same Illumina BeadChip data from published experiments churns out better differentially expressed gene selection after applying our proposed summary statistic.

This was particularly evident in the low concentration range of the Illumina spike experiment [11, 14]. Given that this range is typical for the endogenous gene expression, the improvement should also be observed in biological experiments as well. Indeed, the superior statistical significance contributed markedly to more successful biological pathway elucidations. This was demonstrated with the MT1-MMP [7] data as well as the human spermatogenesis [8] data. For these two examples, more relevant differentially expressed genes were revealed when our proposed summary statistic was applied. In fact, a number of these genes has already been independently validated in the literature [21, 2734]. Their biological significance was demonstrated through functional studies like gene knock-out, mutagenesis and quantification studies like RT-PCR and immunoblotting. Finally in the context of cross-platform comparison between Affymetrix and Illumina, more concordant results were recovered for the spermatogenesis expression profile [8]. This should not be surprising because our summary statistic is a close parallel to that of Affymetrix.

To conclude, our work is most relevant and imperative to any investigator who wants to derive more accurate differentially expressed gene lists from Illumina data.

Materials and methods

The Illumina spike experiment

We exploited the dataset from a published artificial spike experiment [11, 14]; the complete dataset was obtained as a personal communication by Semyon Kruglyak [See additional file 1]. In total, 4 versions for each of eight artificial polyadenylated RNAs (bla, cat, cre, e1a, gfp, gst, gus, lux) were generated by the authors. Although it was not mentioned in [11, 14], the dataset contains two versions of another artificial polyadenylated RNA (neo). Therefore, there are altogether 34 unique labeled and spiked 50 mers against a human cRNA background for each of the spike concentrations. The pooled spike RNAs were tested at a total of twelve different concentrations (0, 0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30, 100, 300, 1000) pM. Each of the spiked and labeled samples (at 1.5 μg per sample) was hybridized in quadruplicates across 48 arrays on eight different Human-6 Expression BeadChips.

The MT1-MMT (membrane type-1 matrix metalloproteinase) experiment

This dataset available as NCBI GEO GSE5095 was complemented with replicate-specific standard errors and number of beads in a private communication by Vladislav S. Golubkov. In the experiment, 184B5 human normal mammary epithelial cells were transfected with MT1-MMP [7]. Total RNA was then isolated from the 184B5-MT and 184B5 cell culture, following DNA-chip RNA expression profiling using Illumina Human-6 Expression BeadChips.

The human spermatogenesis experiment

The published expression profile dataset NCBI GEO GSE6969 from human ejaculates was used [8]. In the experiment, the samples were collected from 17 normal fertile men and 14 teratozoospermic men aged between 21 to 57. Upon RNA isolation of the spermatozoa, RNA expression profiling was carried out on both the Affymetrix and Illumina platform. 4 out of 17 normal and 6 out of 14 teratozoospermic samples are profiled by the Illumina Human-8 Expression BeadChips while the remaining 13 normal and 8 teratozoospermic samples were profiled by the Affymetrix HG-U133 (version 2) GeneChips. In addition, 5 out of these 13 normal samples and the same 8 teratozoospermic samples were profiled again by the Illumina Human-6 Expression BeadChips.

Array normalization procedure

Our proposed procedure is inspired by quantile normalization and by the scaling method used by Affymetrix [16]. The quantile normalization variant is applied to groups of technical replicates with the goal to achieve equal spread in the distribution of the bead intensities for each array. Then, the scaling method is applied to all the arrays to ensure that the medians of all arrays are equal.

For the sake of simplicity, the normalization procedure illustrated below will be based on only one treatment condition. The same steps will be repeated for other treatment conditions. Therefore, an arbitrary gene g from an array consisting of a total of G genes with K technical replicates each will have summary data μg,k, σg,k, Mg,kwhere g = 1,...,G and k = 1,...,K.

Log-transformation is first applied on the mean bead intensities for all readings. The k-th technical replicate of gene g after undergoing log-transformation is depicted as log2(μg,k).

First, normalization within the replicate is performed. For the k-th technical replicate, the median and standard deviation for the log mean bead intensities within replicate are calculated as
m e d i a n log 2 ( μ k , w t r e p ) = m e d i a n [ log 2 ( μ 1 , k ) , ... , log 2 ( μ G , k ) ] σ log 2 ( μ k , w t r e p ) = 1 G g = 1 G [ log 2 ( μ g , k ) m e d i a n log 2 ( μ k , w t r e p ) ] 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabiqaaaqaaiabd2gaTjabdwgaLjabdsgaKjabdMgaPjabdggaHjabd6gaUnaaBaaaleaacyGGSbaBcqGGVbWBcqGGNbWzdaWgaaadbaGaeGOmaidabeaaliabcIcaOiabeY7aTnaaBaaameaacqWGRbWAcqGGSaalcqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaSGaeiykaKcabeaakiabg2da9iabd2gaTjabdwgaLjabdsgaKjabdMgaPjabdggaHjabd6gaUjabcUfaBjGbcYgaSjabc+gaVjabcEgaNnaaBaaaleaacqaIYaGmaeqaaOGaeiikaGIaeqiVd02aaSbaaSqaaiabigdaXiabcYcaSiabdUgaRbqabaGccqGGPaqkcqGGSaalcqGGUaGlcqGGUaGlcqGGUaGlcqGGSaalcyGGSbaBcqGGVbWBcqGGNbWzdaWgaaWcbaGaeGOmaidabeaakiabcIcaOiabeY7aTnaaBaaaleaacqWGhbWrcqGGSaalcqWGRbWAaeqaaOGaeiykaKIaeiyxa0fabaGaeq4Wdm3aaSbaaSqaaiGbcYgaSjabc+gaVjabcEgaNnaaBaaameaacqaIYaGmaeqaaSGaeiikaGIaeqiVd02aaSbaaWqaaiabdUgaRjabcYcaSiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaWccqGGPaqkaeqaaOGaeyypa0ZaaOaaaeaajuaGdaWcaaqaaiabigdaXaqaaiabdEeahbaakmaaqahabaWaamWaaeaacyGGSbaBcqGGVbWBcqGGNbWzdaWgaaWcbaGaeGOmaidabeaakiabcIcaOiabeY7aTnaaBaaaleaacqWGNbWzcqGGSaalcqWGRbWAaeqaaOGaeiykaKIaeyOeI0IaemyBa0MaemyzauMaemizaqMaemyAaKMaemyyaeMaemOBa42aaSbaaSqaaiGbcYgaSjabc+gaVjabcEgaNnaaBaaameaacqaIYaGmaeqaaSGaeiikaGIaeqiVd02aaSbaaWqaaiabdUgaRjabcYcaSiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaWccqGGPaqkaeqaaaGccaGLBbGaayzxaaWaaWbaaSqabeaacqaIYaGmaaaabaGaem4zaCMaeyypa0JaeGymaedabaGaem4raCeaniabggHiLdaaleqaaaaaaaa@B894@
The median-of-medians within replicate and its corresponding standard deviation, hereby depicted as MOM wtrep and σ M O M w t r e p MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aaSbaaSqaaiabd2eanjabd+eapjabd2eannaaBaaameaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaaWcbeaaaaa@387A@ respectively, are given as
σ M O M w t r e p = σ log 2 ( μ k * , w t r e p ) where  k * = arg k [ m e d i a n log 2 ( μ k , w t r e p ) = M O M w t r e p ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabiqaaaqaaiabeo8aZnaaBaaaleaacqWGnbqtcqWGpbWtcqWGnbqtdaWgaaadbaGaem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaaaSqabaGccqGH9aqpcqaHdpWCdaWgaaWcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaWqaaiabikdaYaqabaWccqGGOaakcqaH8oqBdaWgaaadbaGaem4AaSMaeiOkaOIaeiilaWIaem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaaliabcMcaPaqabaaakeaacqqG3bWDcqqGObaAcqqGLbqzcqqGYbGCcqqGLbqzcqqGGaaicqWGRbWAcqGGQaGkcqGH9aqpdaWfqaqaaiGbcggaHjabckhaYjabcEgaNbWcbaGaem4AaSgabeaakiabcUfaBjabd2gaTjabdwgaLjabdsgaKjabdMgaPjabdggaHjabd6gaUnaaBaaaleaacyGGSbaBcqGGVbWBcqGGNbWzdaWgaaadbaGaeGOmaidabeaaliabcIcaOiabeY7aTnaaBaaameaacqWGRbWAcqGGSaalcqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaSGaeiykaKcabeaakiabg2da9iabd2eanjabd+eapjabd2eannaaBaaaleaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaOGaeiyxa0faaaaa@883D@
Therefore, the normalized log mean bead intensities within replicate for the k-th technical replicate of gene g is defined as
log 2 ( μ g , k , n o r m w t r e p ) = [ log ( μ g , k ) m e d i a n log 2 ( μ k , w t r e p ) σ log 2 ( μ k , w t r e p ) ] σ M O M w t r e p + M O M w t r e p MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaSqaaiabikdaYaqabaGccqGGOaakcqaH8oqBdaWgaaWcbaGaem4zaCMaeiilaWIaem4AaSMaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0Maem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaakiabcMcaPiabg2da9maadmaajuaGbaWaaSaaaeaacyGGSbaBcqGGVbWBcqGGNbWzcqGGOaakcqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdUgaRbqabaGaeiykaKIaeyOeI0IaemyBa0MaemyzauMaemizaqMaemyAaKMaemyyaeMaemOBa42aaSbaaeaacyGGSbaBcqGGVbWBcqGGNbWzdaWgaaqaaiabikdaYaqabaGaeiikaGIaeqiVd02aaSbaaeaacqWGRbWAcqGGSaalcqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaiabcMcaPaqabaaabaGaeq4Wdm3aaSbaaeaacyGGSbaBcqGGVbWBcqGGNbWzdaWgaaqaaiabikdaYaqabaGaeiikaGIaeqiVd02aaSbaaeaacqWGRbWAcqGGSaalcqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaiabcMcaPaqabaaaaaGccaGLBbGaayzxaaGaey4fIOIaeq4Wdm3aaSbaaSqaaiabd2eanjabd+eapjabd2eannaaBaaameaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaaWcbeaakiabgUcaRiabd2eanjabd+eapjabd2eannaaBaaaleaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeqaaaaa@9D49@
The median for the normalized log mean bead intensities within the k-th replicate is then re-calculated in a similar fashion as before, where
m e d i a n log 2 ( μ k , n o r m w t r e p ) = m e d i a n [ log 2 ( μ 1 , k , n o r m w t r e p ) , ... , log 2 ( μ G , k , n o r m w t r e p ) ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyBa0MaemyzauMaemizaqMaemyAaKMaemyyaeMaemOBa42aaSbaaSqaaiGbcYgaSjabc+gaVjabcEgaNnaaBaaameaacqaIYaGmaeqaaSGaeiikaGIaeqiVd02aaSbaaWqaaiabdUgaRjabcYcaSiabd6gaUjabd+gaVjabdkhaYjabd2gaTjabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaWccqGGPaqkaeqaaOGaeyypa0JaemyBa0MaemyzauMaemizaqMaemyAaKMaemyyaeMaemOBa4Maei4waSLagiiBaWMaei4Ba8Maei4zaC2aaSbaaSqaaiabikdaYaqabaGccqGGOaakcqaH8oqBdaWgaaWcbaGaeGymaeJaeiilaWIaem4AaSMaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0Maem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaakiabcMcaPiabcYcaSiabc6caUiabc6caUiabc6caUiabcYcaSiGbcYgaSjabc+gaVjabcEgaNnaaBaaaleaacqaIYaGmaeqaaOGaeiikaGIaeqiVd02aaSbaaSqaaiabdEeahjabcYcaSiabdUgaRjabcYcaSiabd6gaUjabd+gaVjabdkhaYjabd2gaTjabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaGccqGGPaqkcqGGDbqxaaa@8FAA@
The median-of-medians across replicates, depicted as MOM acrep , is defined as
M O M a c r e p = m e d i a n [ m e d i a n log 2 ( μ 1 , n o r m w t r e p ) , ... , m e d i a n log 2 ( μ K , n o r m w t r e p ) ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyta0Kaem4ta8Kaemyta00aaSbaaSqaaiabdggaHjabdogaJjabdkhaYjabdwgaLjabdchaWbqabaGccqGH9aqpcqWGTbqBcqWGLbqzcqWGKbazcqWGPbqAcqWGHbqycqWGUbGBcqGGBbWwcqWGTbqBcqWGLbqzcqWGKbazcqWGPbqAcqWGHbqycqWGUbGBdaWgaaWcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaWqaaiabikdaYaqabaWccqGGOaakcqaH8oqBdaWgaaadbaGaeGymaeJaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0Maem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaaliabcMcaPaqabaGccqGGSaalcqGGUaGlcqGGUaGlcqGGUaGlcqGGSaalcqWGTbqBcqWGLbqzcqWGKbazcqWGPbqAcqWGHbqycqWGUbGBdaWgaaWcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaWqaaiabikdaYaqabaWccqGGOaakcqaH8oqBdaWgaaadbaGaem4saSKaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0Maem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaaliabcMcaPaqabaGccqGGDbqxaaa@8616@
Therefore, the normalized log mean bead intensities across replicates will be
log 2 ( μ g , k , n o r m a c r e p ) = log 2 ( μ g , k , n o r m w t r e p ) m e d i a n log 2 ( μ k , w t r e p ) + M O M a c r e p MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaSqaaiabikdaYaqabaGccqGGOaakcqaH8oqBdaWgaaWcbaGaem4zaCMaeiilaWIaem4AaSMaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0MaemyyaeMaem4yamMaemOCaiNaemyzauMaemiCaahabeaakiabcMcaPiabg2da9iGbcYgaSjabc+gaVjabcEgaNnaaBaaaleaacqaIYaGmaeqaaOGaeiikaGIaeqiVd02aaSbaaSqaaiabdEgaNjabcYcaSiabdUgaRjabcYcaSiabd6gaUjabd+gaVjabdkhaYjabd2gaTjabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqabaGccqGGPaqkcqGHsislcqWGTbqBcqWGLbqzcqWGKbazcqWGPbqAcqWGHbqycqWGUbGBdaWgaaWcbaGagiiBaWMaei4Ba8Maei4zaC2aaSbaaWqaaiabikdaYaqabaWccqGGOaakcqaH8oqBdaWgaaadbaGaem4AaSMaeiilaWIaem4DaCNaemiDaqNaemOCaiNaemyzauMaemiCaahabeaaliabcMcaPaqabaGccqGHRaWkcqWGnbqtcqWGpbWtcqWGnbqtdaWgaaWcbaGaemyyaeMaem4yamMaemOCaiNaemyzauMaemiCaahabeaaaaa@8794@
The normalized log mean bead intensities across replicates are then transformed back to the original scale, where
μ g , k , n o r m = 2 log 2 ( μ g , k , n o r m a c r e p ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabdEgaNjabcYcaSiabdUgaRjabcYcaSiabd6gaUjabd+gaVjabdkhaYjabd2gaTbqabaGccqGH9aqpcqaIYaGmdaahaaWcbeqaaiGbcYgaSjabc+gaVjabcEgaNnaaBaaameaacqaIYaGmaeqaaSGaeiikaGIaeqiVd02aaSbaaWqaaiabdEgaNjabcYcaSiabdUgaRjabcYcaSiabd6gaUjabd+gaVjabdkhaYjabd2gaTjabdggaHjabdogaJjabdkhaYjabdwgaLjabdchaWbqabaWccqGGPaqkaaaaaa@5403@

and the corresponding standard deviation is now

σg,k,norm= scale σ * σg,k
where  s c a l e σ = μ g , k , n o r m μ g , k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaeeiiaaIaem4CamNaem4yamMaemyyaeMaemiBaWMaemyzau2aaSbaaSqaaiabeo8aZbqabaGccqGH9aqpdaGcaaqcfayaamaalaaabaGaeqiVd02aaSbaaeaacqWGNbWzcqGGSaalcqWGRbWAcqGGSaalcqWGUbGBcqWGVbWBcqWGYbGCcqWGTbqBaeqaaaqaaiabeY7aTnaaBaaabaGaem4zaCMaeiilaWIaem4AaSgabeaaaaaaleqaaaaa@4F8E@

Therefore, after undergoing the array normalization procedure, an arbitrary gene g from an array consisting of a total of G genes with K technical replicates each will now have the summary data μg,k,norm, σg,k,norm, Mg,kwhere g = 1,..., G and k = 1,...,K.

Assume that one treatment condition is replicated K times in the Beadchip, i.e., k = 1,...,K. When the mean estimates μg,kare being treated as the intensity values xg,k,1, the mean and the incorrect summary statistic of the variance of this treatment condition are given as
μ g = 1 K i = 1 K μ g , k , n o r m σ g 2 = 1 K i = 1 K ( μ g , k μ g ) 2 where n g = K MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabiqaaaqaaiabeY7aTnaaBaaaleaacqWGNbWzaeqaaOGaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqWGlbWsaaGcdaaeWbqaaiabeY7aTnaaBaaaleaacqWGNbWzcqGGSaalcqWGRbWAcqGGSaalcqWGUbGBcqWGVbWBcqWGYbGCcqWGTbqBaeqaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaGcbaqbaeqabeWaaaqaaiabeo8aZnaaDaaaleaacqWGNbWzaeaacqaIYaGmaaGccqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabdUealbaakmaaqahabaGaeiikaGIaeqiVd02aaSbaaSqaaiabdEgaNjabcYcaSiabdUgaRbqabaGccqGHsislcqaH8oqBdaWgaaWcbaGaem4zaCgabeaakiabcMcaPmaaCaaaleqabaGaeGOmaidaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzaugabaGaemOBa42aaSbaaSqaaiabdEgaNbqabaGccqGH9aqpcqWGlbWsaaaaaaaa@6D08@

Note that σ g 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEgaNbqaaiabikdaYaaaaaa@300E@ is equivalent to equation (3).

On the other hand, when no misinterpretation of the summary statistic has occurred, the mean and variance are given as

μ g = median(μg,1,norm,...,μg,K,norm)
σ g 2 = σ g , k * , n o r m 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEgaNbqaaiabikdaYaaakiabg2da9iabeo8aZnaaDaaaleaacqWGNbWzcqGGSaalcqWGRbWAcqGGQaGkcqGGSaalcqWGUbGBcqWGVbWBcqWGYbGCcqWGTbqBaeaacqaIYaGmaaaaaa@3F3C@
n g = Mg,k*
where  k * = arg k [ μ g , k , n o r m = μ g ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaee4DaCNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaeeiiaaIaem4AaSMaeiOkaOIaeyypa0ZaaCbeaeaacyGGHbqycqGGYbGCcqGGNbWzaSqaaiabdUgaRbqabaGccqGGBbWwcqaH8oqBdaWgaaWcbaGaem4zaCMaeiilaWIaem4AaSMaeiilaWIaemOBa4Maem4Ba8MaemOCaiNaemyBa0gabeaakiabg2da9iabeY7aTnaaBaaaleaacqWGNbWzaeqaaOGaeiyxa0faaa@4F7B@

Note that σ g 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEgaNbqaaiabikdaYaaaaaa@300E@ is now equivalent to equation (5).

Statistical test procedure

A pair-wise t-test [35] infers if differences exist between the two populations sampled. Furthermore, given 2 treatment conditions c1 and c2, an arbitrary gene g will have two pair of readings μg,c1, σg,c1, ng,c1 and μg,c2, σg,c2, ng,c2. Then if the two treated samples came from normal populations and if both have equal variances, then the t value for the difference between the two treatment conditions is given as
t = μ g , c 2 μ g , c 1 s μ g , c 2 μ g , c 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiDaqNaeyypa0tcfa4aaSaaaeaacqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqabaGaeyOeI0IaeqiVd02aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIXaqmaeqaaaqaaiabdohaZnaaBaaabaGaeqiVd02aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIYaGmaeqaaiabgkHiTiabeY7aTnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaaaeqaaaaaaaa@4BDA@
where s μ g , c 1 μ g , c 2 = s p 2 n g , c 1 + s p 2 n g , c 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4Cam3aaSbaaSqaaiabeY7aTnaaBaaameaacqWGNbWzcqGGSaalcqWGJbWycqaIXaqmaeqaaSGaeyOeI0IaeqiVd02aaSbaaWqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqabaaaleqaaOGaeyypa0ZaaOaaaeaajuaGdaWcaaqaaiabdohaZnaaDaaabaGaemiCaahabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaaaaGccqGHRaWkjuaGdaWcaaqaaiabdohaZnaaDaaabaGaemiCaahabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGOmaidabeaaaaaaleqaaaaa@524C@ , s p 2 = ( n g , c 1 1 ) σ g , c 1 2 + ( n g , c 2 1 ) σ g , c 2 2 n g , c 1 + n g , c 2 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4Cam3aa0baaSqaaiabdchaWbqaaiabikdaYaaakiabg2da9KqbaoaalaaabaGaeiikaGIaemOBa42aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIXaqmaeqaaiabgkHiTiabigdaXiabcMcaPiabeo8aZnaaDaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabaGaeGOmaidaaiabgUcaRiabcIcaOiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGOmaidabeaacqGHsislcqaIXaqmcqGGPaqkcqaHdpWCdaqhaaqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqaaiabikdaYaaaaeaacqWGUbGBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabigdaXaqabaGaey4kaSIaemOBa42aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIYaGmaeqaaiabgkHiTiabikdaYaaaaaa@62CB@ and the degree of freedom is given as ν = ng,c1 + ng,c2-2. However, if the two treatment samples do not have equal variances, the t value is given as
t = μ g , c 2 μ g , c 1 s μ g , c 1 2 n g , c 1 + s μ g , c 2 2 n g , c 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiDaqNaeyypa0tcfa4aaSaaaeaacqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqabaGaeyOeI0IaeqiVd02aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIXaqmaeqaaaqaamaakaaabaWaaSaaaeaacqWGZbWCdaqhaaqaaiabeY7aTnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaaaeaacqaIYaGmaaaabaGaemOBa42aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIXaqmaeqaaaaacqGHRaWkdaWcaaqaaiabdohaZnaaDaaabaGaeqiVd02aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIYaGmaeqaaaqaaiabikdaYaaaaeaacqWGUbGBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqabaaaaaqabaaaaaaa@5B6F@

The degree of freedom is given as v = ( s μ g , c 1 2 n g , c 1 + s μ g , c 2 2 n g , c 2 ) 2 ( s μ g , c 1 2 n g , c 1 ) 2 n g , c 1 3 + ( s μ g , c 2 2 n g , c 2 ) 2 n g , c 2 3 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemODayNaeyypa0tcfa4aaSaaaeaadaqadaqaamaalaaabaGaem4Cam3aa0baaeaacqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabigdaXaqabaaabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaaaaGaey4kaSYaaSaaaeaacqWGZbWCdaqhaaqaaiabeY7aTnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGOmaidabeaaaeaacqaIYaGmaaaabaGaemOBa42aaSbaaeaacqWGNbWzcqGGSaalcqWGJbWycqaIYaGmaeqaaaaaaiaawIcacaGLPaaadaahaaqabeaacqaIYaGmaaaabaWaaSaaaeaadaqadaqaamaalaaabaGaem4Cam3aa0baaeaacqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabigdaXaqabaaabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaaaaaacaGLOaGaayzkaaWaaWbaaeqabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGymaedabeaacqGHsislcqaIZaWmaaGaey4kaSYaaSaaaeaadaqadaqaamaalaaabaGaem4Cam3aa0baaeaacqaH8oqBdaWgaaqaaiabdEgaNjabcYcaSiabdogaJjabikdaYaqabaaabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGOmaidabeaaaaaacaGLOaGaayzkaaWaaWbaaeqabaGaeGOmaidaaaqaaiabd6gaUnaaBaaabaGaem4zaCMaeiilaWIaem4yamMaeGOmaidabeaacqGHsislcqaIZaWmaaaaaaaa@83DD@ .

This is known as the Welch's t test. For a 2-sided alternate hypothesis H A : μg,c1 μg,c2, reject H0 : μg,c1 = μg,c2 if |t|≥ tα(2),ν. It should be noted that the array normalization procedure has been applied before the statistical treatment.

Reviewers' comments

Reviewer's report 1

I. King Jordan, School of Biology, Georgia Institute of Technology

Wong, Loh and Eisenhaber present a novel statistical method for evaluating gene expression data produced using the Illumina BeadChip technology. The fundamental insight that led to the new statistical method is their appreciation that Affymetrix GeneChip microarrays produce single gene expression measurements, while use of Illumina BeadChips yields mean expression values from dozens of identical probes. Therefore, Illumina BeadChip data must be treated differently. Specifically, when technical replicates are available, the standard deviations across replicates for Illumina BeadChip data are best computed as weighted means of the square of the standard deviations of individual measures. In other words, the standard deviations for data sets with technical replicates should be computed from standard errors of individual Illumina BeadChip measures. When this adjustment is applied to several test data sets, the performance of the Illumina BeadChips improves markedly.

While I am not qualified to evaluate the statistical details of their method, the results of its application to the three test data sets appear to be quite convincing. As such, this work represents an important technical development with direct relevance to any study that uses Illumina BeadChip technology.

One of the measures used by the authors to indicate the success of their statistic is increased concordance between lists of differentially expressed genes uncovered by Illumina BeadChip and Affymetrix GeneChip experiments on a spermatogenesis dataset. However, it would seem that the increased replicates of the Illumina BeadChip technology provides for an inherent advantage over the single-measure technology employed with Affymetrix GeneChips. If this is indeed the case, then one may expect improved performance for the Illumina platform relative to Affymetrix and not merely increased concordance as was demonstrated for the spermatogenesis dataset. Do the authors have any sense, or evidence, as to whether the increased sampling of Illumina provides more resolution than Affymetrix? For instance, are the new pathways identified by the Illumina BeadChip analysis of the spermatogenesis dataset a function of the superiority of the platform? Or are the methods complementary, i.e. does the Affymetrix analysis uncover pathways missed by Illumina irrespective of the use of the statistical innovations introduced herein?

Authors' response

There is no reason to assume a superiority of either platform given the same quality of probe sequence design. For example, one might imagine several Affymetrix chips to be mounted on the same glass slide and to be hybridized simultaneously (to resemble the situation of several beads per array). In this case, both platforms can be used in the mode of exclusion of the batch-specific constant shift error as described in the text.

I am wondering about the availability of their method. The authors conclude that the work is relevant, even imperative, to any investigator looking for differentially expressed genes in Illumina data? How are those investigators to use this method – on a web server, as a BioPerl object, as an R routine?

Authors' response

Presently, our code is implemented in Matlab and be obtained on request. It would be straightforward to implement an R version of it so that it can tie back to the bioconductor package in R. Nevertheless, it should not be difficult for any scientist in the area to modify their existing workflow similar to ours based on the equations presented in this paper just by using σ wtrep as standard deviation.

Reviewer's report 2

Mark J. Dunning, Computational Biology Group, Department of Oncology, University of Cambridge, Cancer Research UK Cambridge Research Institute

In my opinion, Wong et. al is a useful addition to the topic of analysis of Illumina data. Whilst the number of publications using Illumina data are growing rapidly, very few authors have tackled the issue of how such data should be analysed. Wong et al. do a very good job of explaining why the usual statistical tests, such as applied to Affymetrix may not be appropriate for Illumina data and that the extra information provided with an Illumina experiment (i.e. accurate gene-specific variances) can produce a more powerful test. It is especially pleasing to see that they are able to pick up biologically relevant results using the new summary statistic.

The investigation into the performance of σ wtrep is well presented. However, a detail in the re-analysis of Golubkov et al. seems to be missing. In the original paper, genes were filtered using the detection scores obtained from Illumina's software. It does not seem that these scores were supplied in GEO, so were these scores also available to Wong et al. as part of their re-analysis? If not, how did they go about reproducing the filtering performed in Golubkov et al.?

Authors' response

Indeed, the data stored in GEO is insufficient to carry out the calculations both in the paper of Golubkov et al. and in this work. We received the standard errors and number of beads through a private communication from Golubkov et al. Our first re-analysis aimed at repeating the work of Golubkov et al. differed from their approach in two aspects. On the one hand, we had another normalization algorithm (see Methods section); on the other hand, we did not carry out filtering. Just to note, the detection score P is calculated from Z-scores of intensities shifted by the background (intensity of negative control spots) and scaled with its standard deviation. In pairwise comparisons involving the Welch's test using the wrong summary statistic σ μ , the differences of intensities do not depend on their previous correction by a constant background. Regardless of these two differences in our re-analysis, the results are essentially identical to the case of Golubkov et al.: The cell cycle pathway did not appear as significantly regulated.

The results supplied in the paper were enough to convince me that the summary statistic σ wtrep is better than current alternatives. However, I'm afraid I was a bit unsure of the connection between σ wtrep and the normalization method proposed by the authors. Can I still use σ wtrep in my differential expression analysis if I use the usual quantile normalization?

Authors' response

The summary statistic σ wtrep can be used with the usual quantile normalization or any normalization methods. One only has to ensure that the standard deviation σ k of the corresponding μ k be adjusted by a transformation factor i.e. σk(normalized) = Aσ k where A = μ k ( n o r m l i z e d ) μ k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyqaeKaeyypa0tcfa4aaSaaaeaacqaH8oqBdaWgaaqaaiabdUgaRjabcIcaOiabd6gaUjabd+gaVjabdkhaYjabd2gaTjabdYgaSjabdMgaPjabdQha6jabdwgaLjabdsgaKjabcMcaPaqabaaabaGaeqiVd02aaSbaaeaacqWGRbWAaeqaaaaaaaa@431B@ . After which, σ wtrep is computed using equation ( 4 ).

What motivated the authors to propose this method of normalization? However, I feel that the description of the normalization procedure was not that easy to follow and would benefit from a small worked example if possible. Do the authors plan to make any of the methodology presented in the paper available in open-source software?

Authors' response

In a typical Illumina BeadChip experiment, different treatment conditions should be hybridized within a chip, while their corresponding technical replicates should be distributed across chips. The treatment conditions within a chip shall be exposed to similar systematic and random error. Hence, the differences in spreads among the arrays or treatment conditions should ideally reflect true biological differences. The motivation of our normalization method is to create a two-step normalization procedure whereby the first step forces the same median and spread only among technical replicates while the second step simply ensures that the medians across all the arrays are common. As such, the spreads among the various treatment conditions need not be the same, thus preserving true differences. The software (as a Matlab program) is available on request.

Aside from these questions, and suggestions to improve readability supplied separately to the authors, I am happy for this manuscript to be published.

Specific Comments for the authors:

• Bottom of Page 2: "But instead of delivering the individual bead intensities, the mean and standard error (i.e. the standard deviation divided by the square root of the number of beads) of the bead intensities, known as the summary data, are reported."

This statement is possibly a bit misleading as the individual bead intensities are available with appropriate scanner modifications (see Dunning et. al). I suggest this statement be changed to acknowledge this, although the summary data are usually the starting points for analysis when using Illumina's software.

• Page 3 paragraph 3 – "Furthermore, this summary statistic will..." This should be changed to either "this summary statistic" or "these summary statistic". I suggest the manuscript be checked for other similar errors.

• Equation 1 should have the sum going from k = 1..K rather than i = 1..K

• Page 5 Paragraph 1 – The weights used in Dunning et. al are the inverse of σ k 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdUgaRbqaaiabikdaYaaaaaa@3016@ rather than σ k .

• Page 5 Paragraph 3 -"Due to the multiple copies of the same probe within a single Illumina array, the standard variation can be computed...."

Should be standard deviation rather than standard variation?

  • Page 6 Paragraph 4 – "On the other hand, a trend of growth of mean estimates σ μ .."

Should this be standard deviation of mean estimates?

Authors' response

The suggested amendments have been made accordingly.

Reviewer's report 3

Shamil Sunyaev, Division of Genetics, Dept. of Medicine, Brigham & Women's Hospital and Harvard Medical School

This manuscript describes a new statistical method for the analysis of Illumina BeadChip microarrays. The authors realized that variance is underestimated for these microarrays because the measurements themselves are averaged over multiple probes. Thus, they suggest a new estimate of variance to be used in the analysis based on the Welch's t-test.

The manuscript is well written. The statistical approach is straightforward but has been convincingly demonstrated to produce biologically meaningful results. The authors show that the corrected standard deviation estimate helps obtaining better results for the spike dataset (Chudin et al.) and also reveal more biologically relevant genes in the human spermatogenesis dataset and mammary epithelial dataset.

As an outsider in this field, I do not understand why the analysis is based on the t-test, which heavily depends on sample estimates of variance and assumes normality. It seems that non-parametric method may suite the problem better.

Authors' response

First of all, it is important to note that the summary data is computed using the arithmetic mean and the standard deviation formulas. These formulas are the maximum likelihood estimates (MLEs) of the normal distribution. In other words, normality is innately assumed on the summary data. Furthermore, the typical sample size for each gene measurement in Illumina is about 30 and t-test is known to be robust when sample size is large. More notably, t-test is robust against assumption violations as long as the sample sizes are almost equal and that only two-tailed hypotheses are considered. These were the conditions for all our test cases.

Appendix 1

Proof for σ t o t a l 2 = σ w t r e p 2 + σ μ 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqaaiabikdaYaaakiabg2da9iabeo8aZnaaDaaaleaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeaacqaIYaGmaaGccqGHRaWkcqaHdpWCdaqhaaWcbaGaeqiVd0gabaGaeGOmaidaaaaa@4633@

With reference to Table 1, if the individual bead intensities are available, the mean μ total and squared standard deviation σ t o t a l 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqaaiabikdaYaaaaaa@35AC@ of the total set of bead intensities across K technical replicates can be computed as
μ t o t a l = C i = 1 K n = 1 M k x k , n MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaSbaaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqabaGccqGH9aqpcqWGdbWqcqGHflY1daaeWbqaamaaqahabaGaemiEaG3aaSbaaSqaaiabdUgaRjabcYcaSiabd6gaUbqabaaabaGaemOBa4Maeyypa0JaeGymaedabaGaemyta00aaSbaaWqaaiabdUgaRbqabaaaniabggHiLdaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoaaaa@4D9F@
(8)
σ t o t a l 2 = C i = 1 K n = 1 M k x k , n 2 μ t o t a l 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqaaiabikdaYaaakiabg2da9iabdoeadjabgwSixpaaqahabaWaaabCaeaacqWG4baEdaqhaaWcbaGaem4AaSMaeiilaWIaemOBa4gabaGaeGOmaidaaaqaaiabd6gaUjabg2da9iabigdaXaqaaiabd2eannaaBaaameaacqWGRbWAaeqaaaqdcqGHris5aaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdGccqGHsislcqaH8oqBdaqhaaWcbaGaemiDaqNaem4Ba8MaemiDaqNaemyyaeMaemiBaWgabaGaeGOmaidaaaaa@5A53@
(9)
where, for convenience of notation, we introduce
C = ( k = 1 K M k ) 1 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4qamKaeyypa0ZaaeWaaeaadaaeWbqaaiabd2eannaaBaaaleaacqWGRbWAaeqaaaqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaGccaGLOaGaayzkaaWaaWbaaSqabeaacqGHsislcqaIXaqmaaaaaa@3B2E@
(10)
On the other hand, only the summary data are given. Then the mean μ total and the squared standard deviation σ μ 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabikdaYaaaaaa@306D@ of the average bead intensities across K technical replicates are given as
μ = C i = 1 K M k μ k = C i = 1 K n = 1 M k x k , n MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd0Maeyypa0Jaem4qam0aaabCaeaacqWGnbqtdaWgaaWcbaGaem4AaSgabeaakiabeY7aTnaaBaaaleaacqWGRbWAaeqaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOGaeyypa0Jaem4qam0aaabCaeaadaaeWbqaaiabdIha4naaBaaaleaacqWGRbWAcqGGSaalcqWGUbGBaeqaaaqaaiabd6gaUjabg2da9iabigdaXaqaaiabd2eannaaBaaameaacqWGRbWAaeqaaaqdcqGHris5aaWcbaGaemyAaKMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaaaa@52E9@
(11)
σ μ 2 = C k = 1 K M k μ k 2 [ C k = 1 K M k μ k ] 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabikdaYaaakiabg2da9iabdoeadnaaqahabaGaemyta00aaSbaaSqaaiabdUgaRbqabaGccqaH8oqBdaqhaaWcbaGaem4AaSgabaGaeGOmaidaaaqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOGaeyOeI0YaamWaaeaacqWGdbWqdaaeWbqaaiabd2eannaaBaaaleaacqWGRbWAaeqaaOGaeqiVd02aaSbaaSqaaiabdUgaRbqabaaabaGaem4AaSMaeyypa0JaeGymaedabaGaem4saSeaniabggHiLdaakiaawUfacaGLDbaadaahaaWcbeqaaiabikdaYaaaaaa@5236@
(12)
Clearly, the weighted average of the K bead intensity averages is equal to μ total (equation 8 and 11). For later usage, the equation (12) can be transformed by representing μ k by the average of actual bead intensities M k 1 n = 1 M k x k , n MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyta00aa0baaSqaaiabdUgaRbqaaiabgkHiTiabigdaXaaakmaaqahabaGaemiEaG3aaSbaaSqaaiabdUgaRjabcYcaSiabd6gaUbqabaaabaGaemOBa4Maeyypa0JaeGymaedabaGaemyta00aaSbaaWqaaiabdUgaRbqabaaaniabggHiLdaaaa@3DF5@ .
σ μ 2 = C k = 1 K M k 1 ( n = 1 M k x k , n ) 2 C 2 ( k = 1 K n = 1 M k x k , n ) 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabikdaYaaakiabg2da9iabdoeadjabgwSixpaaqahabaGaemyta00aa0baaSqaaiabdUgaRbqaaiabgkHiTiabigdaXaaaaeaacqWGRbWAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoakmaabmaabaWaaabCaeaacqWG4baEdaWgaaWcbaGaem4AaSMaeiilaWIaemOBa4gabeaaaeaacqWGUbGBcqGH9aqpcqaIXaqmaeaacqWGnbqtdaWgaaadbaGaem4AaSgabeaaa0GaeyyeIuoaaOGaayjkaiaawMcaamaaCaaaleqabaGaeGOmaidaaOGaeyOeI0Iaem4qam0aaWbaaSqabeaacqaIYaGmaaGcdaqadaqaamaaqahabaWaaabCaeaacqWG4baEdaWgaaWcbaGaem4AaSMaeiilaWIaemOBa4gabeaaaeaacqWGUbGBcqGH9aqpcqaIXaqmaeaacqWGnbqtdaWgaaadbaGaem4AaSgabeaaa0GaeyyeIuoaaSqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaGccaGLOaGaayzkaaWaaWbaaSqabeaacqaIYaGmaaaaaa@6AC2@
(13)
On the other hand, the variance in bead intensities across K technical replicates σ w t r e p 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqaaiabikdaYaaaaaa@35C8@ is defined as weighted average
σ w t r e p 2 = C k = 1 K M k σ k 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqaaiabikdaYaaakiabg2da9iabdoeadjabgwSixpaaqahabaGaemyta00aaSbaaSqaaiabdUgaRbqabaGccqaHdpWCdaqhaaWcbaGaem4AaSgabaGaeGOmaidaaaqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaaa@4823@
(14)
As before, the values σ k 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdUgaRbqaaiabikdaYaaaaaa@3016@ are substituted by their definition in terms of actual measurements. Thus, we obtain for σ w t r e p 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqaaiabikdaYaaaaaa@35C8@
σ w t r e p 2 = C k = 1 K M k [ M k 1 n = 1 M k x k , n 2 ( M k 1 n = 1 M k x k , n ) 2 ] MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqaaiabikdaYaaakiabg2da9iabdoeadjabgwSixpaaqahabaGaemyta00aaSbaaSqaaiabdUgaRbqabaGcdaWadaqaaiabd2eannaaDaaaleaacqWGRbWAaeaacqGHsislcqaIXaqmaaGcdaaeWbqaaiabdIha4naaDaaaleaacqWGRbWAcqGGSaalcqWGUbGBaeaacqaIYaGmaaaabaGaemOBa4Maeyypa0JaeGymaedabaGaemyta00aaSbaaWqaaiabdUgaRbqabaaaniabggHiLdGccqGHsisldaqadaqaaiabd2eannaaDaaaleaacqWGRbWAaeaacqGHsislcqaIXaqmaaGcdaaeWbqaaiabdIha4naaBaaaleaacqWGRbWAcqGGSaalcqWGUbGBaeqaaaqaaiabd6gaUjabg2da9iabigdaXaqaaiabd2eannaaBaaameaacqWGRbWAaeqaaaqdcqGHris5aaGccaGLOaGaayzkaaWaaWbaaSqabeaacqaIYaGmaaaakiaawUfacaGLDbaaaSqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aaaa@6EC5@
(15)
Minor transformations yield the equation
σ w t r e p 2 = C k = 1 K n = 1 M k x k , n 2 C k = 1 K M k 1 ( n = 1 M k x k , n ) 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdEha3jabdsha0jabdkhaYjabdwgaLjabdchaWbqaaiabikdaYaaakiabg2da9iabdoeadjabgwSixpaaqahabaWaaabCaeaacqWG4baEdaqhaaWcbaGaem4AaSMaeiilaWIaemOBa4gabaGaeGOmaidaaaqaaiabd6gaUjabg2da9iabigdaXaqaaiabd2eannaaBaaameaacqWGRbWAaeqaaaqdcqGHris5aOGaeyOeI0Iaem4qamKaeyyXIC9aaabCaeaacqWGnbqtdaqhaaWcbaGaem4AaSgabaGaeyOeI0IaeGymaedaaOWaaeWaaeaadaaeWbqaaiabdIha4naaBaaaleaacqWGRbWAcqGGSaalcqWGUbGBaeqaaaqaaiabd6gaUjabg2da9iabigdaXaqaaiabd2eannaaBaaameaacqWGRbWAaeqaaaqdcqGHris5aaGccaGLOaGaayzkaaaaleaacqWGRbWAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoaaSqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOWaaWbaaSqabeaacqaIYaGmaaaaaa@6F94@
(16)

Obviously, the second term in equation (16) and the first in equation (13) are identical except for the sign. Together with equations (8) and (9), this proves that σ t o t a l 2 = σ w t r e p 2 + σ μ 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdsha0jabd+gaVjabdsha0jabdggaHjabdYgaSbqaaiabikdaYaaakiabg2da9iabeo8aZnaaDaaaleaacqWG3bWDcqWG0baDcqWGYbGCcqWGLbqzcqWGWbaCaeaacqaIYaGmaaGccqGHRaWkcqaHdpWCdaqhaaWcbaGaeqiVd0gabaGaeGOmaidaaaaa@4633@ . Note that, in practice, the number of beads for each replicate is roughly equal. Hence when M k M for k = 1,...,K, the weighted arithmetic averages in the above equations can be justifiably by normal averages.

Appendix 2

Computation of σ total under the condition of no systematic error

We assume the k-th batch-specific expression vector X ¯ k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmiwaGLbaebadaWgaaWcbaGaem4AaSgabeaaaaa@2EB1@ in Table 1 to be composed of a random, batch-independent component Y ¯ k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmywaKLbaebadaWgaaWcbaGaem4AaSgabeaaaaa@2EB3@ (with standard deviation σ k and batch-independent mean μ) and batch-specific systematic shift vector S ¯ k MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafm4uamLbaebadaWgaaWcbaGaem4AaSgabeaaaaa@2EA7@ having equal components s k (thus, with mean s k and zero standard deviation). Whereas the standard deviation of the k-th replicate is not affected by the constant systematic error, the mean μ k is given as μ k = μ + s k . Therefore, the mean μ sys across replicates is
μ s y s = 1 K i = 1 K ( μ + s k ) = μ + 1 K i = 1 K s k = μ + s MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiVd02aaWbaaSqabeaacqWGZbWCcqWG5bqEcqWGZbWCaaGccqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabdUealbaakmaaqahabaGaeiikaGIaeqiVd0galeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoakiabgUcaRiabdohaZnaaBaaaleaacqWGRbWAaeqaaOGaeiykaKIaeyypa0JaeqiVd0Maey4kaSscfa4aaSaaaeaacqaIXaqmaeaacqWGlbWsaaGcdaaeWbqaaiabdohaZnaaBaaaleaacqWGRbWAaeqaaaqaaiabdMgaPjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOGaeyypa0JaeqiVd0Maey4kaSIaem4Camhaaa@593D@
(17)
where s denotes the average of the systematic shifts. The standard deviation σ μ s y s MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabdohaZjabdMha5jabdohaZbaaaaa@33D4@ of the means from the K replicates is essentially the standard deviation σ s of the systematic shifts as is shown with the derivation
( σ μ s y s ) 2 = 1 K k = 1 K ( μ k μ s y s ) 2 = 1 K k = 1 K ( s k s ) 2 = σ s 2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeWabiqaaaqaamaabmaabaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabdohaZjabdMha5jabdohaZbaaaOGaayjkaiaawMcaamaaCaaaleqabaGaeGOmaidaaOGaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqWGlbWsaaGcdaaeWbqaaiabcIcaOiabeY7aTnaaBaaaleaacqWGRbWAaeqaaOGaeyOeI0IaeqiVd02aaWbaaSqabeaacqWGZbWCcqWG5bqEcqWGZbWCaaGccqGGPaqkdaahaaWcbeqaaiabikdaYaaaaeaacqWGRbWAcqGH9aqpcqaIXaqmaeaacqWGlbWsa0GaeyyeIuoaaOqaaiabg2da9KqbaoaalaaabaGaeGymaedabaGaem4saSeaaOWaaabCaeaacqGGOaakcqWGZbWCdaWgaaWcbaGaem4AaSgabeaakiabgkHiTiabdohaZjabcMcaPmaaCaaaleqabaGaeGOmaidaaaqaaiabdUgaRjabg2da9iabigdaXaqaaiabdUealbqdcqGHris5aOGaeyypa0Jaeq4Wdm3aa0baaSqaaiabdohaZbqaaiabikdaYaaaaaaaaa@66A8@
(18)

Consequently, the standard deviation of bead intensities σ μ s y s MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabeY7aTbqaaiabdohaZjabdMha5jabdohaZbaaaaa@33D4@ is plagued by the systematic error. On the other hand, the standard deviation σ k of the k-th replicate is not affected by the batch-specific shift and, therefore, the batch-specific systematic error does not affect σ wtrep . Rightfully, the systematic error should not be present after array normalization. Hence, under the assumption of no systematic error after array normalization, usage of σ wtrep as reliable (lower) estimate of σ total is justified.

List of abbreviations

FP: 

false positive

MT1-MMT: 

membrane type-1 matrix metalloproteinase

TP: 

true positive.

Declarations

Acknowledgements

The authors are grateful to Semyon Kruglyak for providing the Illumina spike dataset; to Vladislav S. Golubkov for providing the complete dataset of the MT1-MMP study; to Adrian E. Platts for providing the significant gene lists derived from Illumina platforms on the Human spermatogenesis study.

Authors’ Affiliations

(1)
Bioinformatics Institute (BII), Agency for Science, Technology and Research (A*STAR)

References

  1. Barnes M, Freudenberg J, Thompson S, Aronow B, Pavlidis P: Experimental comparison and cross-validation of the Affymetrix and Illumina gene expression analysis platforms. Nucleic Acids Res. 2005, 33: 5914-5923. 10.1093/nar/gki890.PubMedPubMed CentralView ArticleGoogle Scholar
  2. Larkin JE, Frank BC, Gavras H, Sultana R, Quackenbush J: Independence and reproducibility across microarray platforms. Nat Methods. 2005, 2: 337-344. 10.1038/nmeth757.PubMedView ArticleGoogle Scholar
  3. Lee JK, Bussey KJ, Gwadry FG, Reinhold W, Riddick G, Pelletier SL, Nishizuka S, Szakacs G, Annereau JP, Shankavaram U, Lababidi S, Smith LH, Gottesman MM, Weinstein JN: Comparing cDNA and oligonucleotide array data: concordance of gene expression across platforms for the NCI-60 cancer cells. Genome Biol. 2003, 4: R82-10.1186/gb-2003-4-12-r82.PubMedPubMed CentralView ArticleGoogle Scholar
  4. Mecham BH, Klus GT, Strovel J, Augustus M, Byrne D, Bozso P, Wetmore DZ, Mariani TJ, Kohane IS, Szallasi Z: Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res. 2004, 32: e74-10.1093/nar/gnh071.PubMedPubMed CentralView ArticleGoogle Scholar
  5. Carter SL, Eklund AC, Mecham BH, Kohane IS, Szallasi Z: Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics. 2005, 6: 107-10.1186/1471-2105-6-107.PubMedPubMed CentralView ArticleGoogle Scholar
  6. Cheadle C, Becker KG, Cho-Chung YS, Nesterova M, Watkins T, Wood W, Prabhu V, Barnes KC: A rapid method for microarray cross platform comparisons using gene expression signatures. Mol Cell Probes. 2007, 21: 35-46. 10.1016/j.mcp.2006.07.004.PubMedView ArticleGoogle Scholar
  7. Golubkov VS, Chekanov AV, Savinov AY, Rozanov DV, Golubkova NV, Strongin AY: Membrane type-1 matrix metalloproteinase confers aneuploidy and tumorigenicity on mammary epithelial cells. Cancer Res. 2006, 66: 10460-10465. 10.1158/0008-5472.CAN-06-2997.PubMedView ArticleGoogle Scholar
  8. Platts AE, Dix DJ, Chemes HE, Thompson KE, Goodrich R, Rockett JC, Rawe VY, Quintana S, Diamond MP, Strader LF, Krawetz SA: Success and failure in human spermatogenesis as revealed by teratozoospermic RNAs. Hum Mol Genet. 2007, 16: 763-773. 10.1093/hmg/ddm012.PubMedView ArticleGoogle Scholar
  9. Bruce SJ, Gardiner BB, Burke LJ, Gongora MM, Grimmond SM, Perkins AC: Dynamic transcription programs during ES cell differentiation towards mesoderm in serum versus serum-freeBMP4 culture. BMC Genomics. 2007, 8: 365-10.1186/1471-2164-8-365.PubMedPubMed CentralView ArticleGoogle Scholar
  10. Liu Y, Shin S, Zeng X, Zhan M, Gonzalez R, Mueller FJ, Schwartz CM, Xue H, Li H, Baker SC, Chudin E, Barker DL, McDaniel TK, Oeser S, Loring JF, Mattson MP, Rao MS: Genome wide profiling of human embryonic stem cells (hESCs), their derivatives and embryonal carcinoma cells to develop base profiles of U.S. Federal government approved hESC lines. BMC Dev Biol. 2006, 6: 20-10.1186/1471-213X-6-20.PubMedPubMed CentralView ArticleGoogle Scholar
  11. Whole-Genome expression analysis using the Sentrix Human-6 and HumanRef-8 expression BeadChips. 2006, Illumina, Inc., [http://www.illumina.com/pagesnrn.ilmn?ID=70#53]
  12. Dunning MJ, Barbosa-Morais NL, Lynch AG, Tavare S, Ritchie ME: Statistical issues in the analysis of Illumina data. BMC Bioinformatics. 2008, 9: 85-10.1186/1471-2105-9-85.PubMedPubMed CentralView ArticleGoogle Scholar
  13. Lin SM, Du P, Huber W, Kibbe WA: Model-based variance-stabilizing transformation for Illumina microarray data. Nucleic Acids Res. 2008, 36: e11-10.1093/nar/gkm1075.PubMedPubMed CentralView ArticleGoogle Scholar
  14. Chudin E, Kruglyak S, Baker SC, Oeser S, Barker D, McDaniel TK: A model of technical variation of microarray signals. J Comput Biol. 2006, 13: 996-1003. 10.1089/cmb.2006.13.996.PubMedView ArticleGoogle Scholar
  15. Scheweder T, Spojotvoll E: Plots of p-values to evaluate many tests simultaneously. Biometrika. 1982, 69: 493-502.View ArticleGoogle Scholar
  16. Bolstad BM, Irizarry RA, Astrand M, Speed TP: A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics. 2003, 19: 185-193. 10.1093/bioinformatics/19.2.185.PubMedView ArticleGoogle Scholar
  17. Dennis G, Sherman BT, Hosack DA, Yang J, Gao W, Lane HC, Lempicki RA: DAVID: Database for Annotation, Visualization, and Integrated Discovery. Genome Biol. 2003, 4: 3-10.1186/gb-2003-4-5-p3.View ArticleGoogle Scholar
  18. Hosack DA, Dennis G, Sherman BT, Lane HC, Lempicki RA: Identifying biological themes within lists of genes with EASE. Genome Biol. 2003, 4: R70-10.1186/gb-2003-4-10-r70.PubMedPubMed CentralView ArticleGoogle Scholar
  19. Li C, Wong WH: Model-based analysis of oligonucleotide arrays: expression index computation and outlier detection. Proc Natl Acad Sci U S A. 2001, 98: 31-36. 10.1073/pnas.011404098.PubMedPubMed CentralView ArticleGoogle Scholar
  20. A.Hess R: Spermatogenesis overview. Encyclopedia of Reproduction. 1999, San Diego, Academic Press, 539-545.Google Scholar
  21. Fukuda MN, Akama TO: In vivo role of alpha-mannosidase IIx: ineffective spermatogenesis resulting from targeted disruption of the Man2a2 in the mouse. Biochim Biophys Acta. 2002, 1573: 382-387.PubMedView ArticleGoogle Scholar
  22. Fukuda MN, Akama TO: The role of N-glycans in spermatogenesis. Cytogenet Genome Res. 2003, 103: 302-306. 10.1159/000076817.PubMedView ArticleGoogle Scholar
  23. Fukuda MN, Akama TO: The in vivo role of alpha-mannosidase IIx and its role in processing of N-glycans in spermatogenesis. Cell Mol Life Sci. 2003, 60: 1351-1355. 10.1007/s00018-003-2339-x.PubMedView ArticleGoogle Scholar
  24. Yan HH, Mruk DD, Cheng CY: Junction restructuring and spermatogenesis: the biology, regulation, and implication in male contraceptive development. Curr Top Dev Biol. 2008, 80: 57-92.PubMedView ArticleGoogle Scholar
  25. Lui WY, Mruk D, Lee WM, Cheng CY: Sertoli cell tight junction dynamics: their regulation during spermatogenesis. Biol Reprod. 2003, 68: 1087-1097. 10.1095/biolreprod.102.010371.PubMedView ArticleGoogle Scholar
  26. Wong EW, Mruk DD, Cheng CY: Biology and regulation of ectoplasmic specialization, an atypical adherens junction type, in the testis. Biochim Biophys Acta. 2007Google Scholar
  27. Shi X, Amindari S, Paruchuru K, Skalla D, Burkin H, Shur BD, Miller DJ: Cell surface beta-1,4-galactosyltransferase-I activates G protein-dependent exocytotic signaling. Development. 2001, 128: 645-654.PubMedGoogle Scholar
  28. Chen Z, Morris C, Allen CM: Changes in dehydrodolichyl diphosphate synthase during spermatogenesis in the rat. Arch Biochem Biophys. 1988, 266: 98-110. 10.1016/0003-9861(88)90240-8.PubMedView ArticleGoogle Scholar
  29. Chen Z, Romrell LJ, Allen CM: Dehydrodolichyl diphosphate synthase in Sertoli and spermatogenic cells of prepuberal rats. J Biol Chem. 1989, 264: 3849-3853.PubMedGoogle Scholar
  30. Gye MC: Expression of claudin-1 in mouse testis. Arch Androl. 2003, 49: 271-279. 10.1080/713828170.PubMedView ArticleGoogle Scholar
  31. Escalier D, Silvius D, Xu X: Spermatogenesis of mice lacking CK2alpha': failure of germ cell survival and characteristic modifications of the spermatid nucleus. Mol Reprod Dev. 2003, 66: 190-201. 10.1002/mrd.10346.PubMedView ArticleGoogle Scholar
  32. Lee NP, Mruk D, Lee WM, Cheng CY: Is the cadherin/catenin complex a functional unit of cell-cell actin-based adherens junctions in the rat testis?. Biol Reprod. 2003, 68: 489-508. 10.1095/biolreprod.102.005793.PubMedView ArticleGoogle Scholar
  33. Lui WY, Lee WM, Cheng CY: Transforming growth factor-beta3 perturbs the inter-Sertoli tight junction permeability barrier in vitro possibly mediated via its effects on occludin, zonula occludens-1, and claudin-11. Endocrinology. 2001, 142: 1865-1877. 10.1210/en.142.5.1865.PubMedGoogle Scholar
  34. Mirza M, Hreinsson J, Strand ML, Hovatta O, Soder O, Philipson L, Pettersson RF, Sollerbrant K: Coxsackievirus and adenovirus receptor (CAR) is expressed in male germ cells and forms a complex with the differentiation factor JAM-C in mouse testis. Exp Cell Res. 2006, 312: 817-830. 10.1016/j.yexcr.2005.11.030.PubMedView ArticleGoogle Scholar
  35. Zar JH: Biostatistical analysis. 1999, Prentice Hall International, Inc., 4thGoogle Scholar

Copyright

© Wong et al; licensee BioMed Central Ltd. 2008

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.