What is a good chi value?

[sammahdi]

I'm having difficulty trying to ascertain what good chi values are, or what range of chi values are acceptable.

Let's say you have a SAXS profile that is a combination of 5 different conformations. This being a linear combination of conformations, there can be multiple solutions with similar chi values at times.

In one instance (say conformation A-E fractions are 0.1,0.5,0.1,0.2,0.1) the chi is 1.2305
In another instance (A-E fractions are 0.2, 0.2, 0.1, 0.5, 0.0) the chi is 1.2310

Different solutions, with similar chi values. Technically the first instance is the global minima, but the different between the 1st and 2nd instance in terms of chi is quite small. Which one is correct? Which one should be chosen?

Furthermore, I've seen papers with chi2 values of 2.1 and call that a "good fit". I've seen posts on here that state chi2 fits of 1.8 are not good fits. I'm having difficulty understanding what constitutes a good chi value.

In another instance for example, the global minima chi is 2.15. Is this considered a poor fit? I've seen papers publish fits with chi2 this high, so I know it's publishable, but I don't know if in terms of "best practices" this is considered a good fit (and thus the results from this fit can be intreperted properly).

[biosaxs.com]

Check out the YouTube video SAXS: what is a good fit? / How to run CRYSOL - it briefly explains what a good chi-squared value is (and what a correlation map test is).

[sammahdi]

Thank you for the link, this answers my 2nd question (albeit I would argue I've seen published SAXS papers that deviate quite largely from the narrow margins presented there, so I presume that is an industry "best practices" but deviations outside those margins are still acceptable?).

But it does not answer my first question. Assume you have a model that fits under the "best fits" category, say n=500 and chi2= 1.15 vs. another fit that is n=500 and chi2=1.14. Which one would you take? Again, these 2 fits have wildly different results and interpretations but they are both considered "good fits".

[AL]

Different solutions, with similar chi values. Technically the first instance is the global minima, but the different between the 1st and 2nd instance in terms of chi is quite small. Which one is correct? Which one should be chosen?

In some cases two or more solutions are equally probable - because of ambiguity of SAXS data. Sometimes adding restraints to the models may resolve the ambiguity, sometimes you just have to accept multiple possible models (or ensembles of models).

Furthermore, I've seen papers with chi2 values of 2.1 and call that a "good fit". I've seen posts on here that state chi2 fits of 1.8 are not good fits. I'm having difficulty understanding what constitutes a good chi value.

In another instance for example, the global minima chi is 2.15. Is this considered a poor fit? I've seen papers publish fits with chi2 this high, so I know it's publishable, but I don't know if in terms of "best practices" this is considered a good fit (and thus the results from this fit can be intreperted properly).

If according to the correlation map test, there are no systematic deviations between the fit and the experimental data and the chi² value is 2.1 then the experimental errors might be underestimated. But in many cases, it is just too challenging to get a perfect fit with a chi² value of 1.0 - because the sample is never ideal (presence of unspecific aggregates, oligomeric polydispersity, conformational polydispersity, repulsive interactions etc.).

[sammahdi]

When you're discussing a correlation map test, I presume (and based on the paper that was linked), you're generating a correlation matrix between your theoretical fits and experimental data. But a follow up on this is what correlation value is considered good? And what section is the correlation matrix generated for? For example I notice you don't generate a correlation matrix using the entire fit, but certain sections.

For example, in one scenario I have a fit that has a chi2 of 1.24, with a correlation of 0.995, but another scenario with a chi2 of 2.20 with a correlation of 0.997. These correlations are for the entire fit. However, if I break it up into sections, the fit with a chi2 of 2.20, the intensities of the first 20 have a correlation of 0.586, the next 20 0.878, the next 20 0.976 (this can of course be visually observed via the residuals, with high residuals at the start and basically zero after). Clearly certain sections have poor correlation which is why the chi2 is high, but the correlation looking at the entire fit is pretty good. Just trying to understand how to intepret all of this and justify my fits and values (the high residuals in the early part of the fit is true of all my fits, they all have high residuals at early q).

[AL]

When you're discussing a correlation map test, I presume (and based on the paper that was linked), you're generating a correlation matrix between your theoretical fits and experimental data. But a follow up on this is what correlation value is considered good?

A p-value > 0.05 is considered good.

And what section is the correlation matrix generated for? For example I notice you don't generate a correlation matrix using the entire fit, but certain sections.

I'm not sure I follow. Typically you want to exclude the lowest angles (where the primary beam is, exclude everything before the first Guinier point) and your data are typically fitted up to a certain q_max (which depends on the quality of your data, the abilities of the program you use for fitting etc.)
Note that the correlation map p-value is calculated based on two values: the length of the longest consecutively different interval and the total number of points in the data sets being compared.

Please note that the noisier your experimental data are the more different models you can fit to it with the same chi² value (or same correlation map p-value).