Chi-Squared Values

Ab initio modelling: DAMMIF, DAMMIN, GASBOR, MONSA
Post Reply
Message
Author
sazhns
Guest
Posts: 2
Joined: 2016.02.15 20:16

Chi-Squared Values

#1 Post by sazhns » 2016.02.16 03:26

Hi all,

I have a quick question on chi-squared values in programs such as CRYSOL and DAMMIN/F. In the literature associated with these papers, chi-squared is calculated as:

X2= (1/N-1) * sum [((( Iobs(s) - Icalc(s))2)/sigma(s)2]

therefore, this means if Iobs=Icalc, chi-squared should be zero. However, reading around, most posts on here suggest a chi-squared value of 1 is better. Where does this come from? Have I misinterpreted something?

ckerr
Active member
Posts: 86
Joined: 2015.03.25 09:03
Location: EMBL Hamburg

Re: Chi-Squared Values

#2 Post by ckerr » 2016.02.16 17:52

Chi squared much less than one suggests that either you have been overfitting the data or your error estimates are too large.

sazhns
Guest
Posts: 2
Joined: 2016.02.15 20:16

Re: Chi-Squared Values

#3 Post by sazhns » 2016.02.22 18:42

Is this quoted anywhere in the literature?

I understand that zero chi-squared is probably due to overfitting and large errors, but it would also be the result of perfect fitting to a curve of little errors (i.e correct solution), would it not?

ckerr
Active member
Posts: 86
Joined: 2015.03.25 09:03
Location: EMBL Hamburg

Re: Chi-Squared Values

#4 Post by ckerr » 2016.02.24 12:50

The chi-square test compares the discrepancies between your theoretical and experimental curves with the expected errors. The null hypothesis is that the discrepancies are entirely due to random experimental errors. A reduced chi-squared value much less than one means that the discrepancies are much smaller than you expected, based on your estimate of what the experimental errors will be. Therefore either the discrepancies really are too small (i.e. overfitting) or the experimental errors are smaller than you estimated.

remba
Member
Posts: 3
Joined: 2019.05.24 15:12

Re: Chi-Squared Values

#5 Post by remba » 2019.05.24 15:14

Hello, I have more general questions:

1) why is there the 1/(k-1) factor?

2) how is calculated the scale factor? can you give me the explicit formula?

Lina PhD student
Member
Posts: 5
Joined: 2020.02.28 11:46

Re: Chi-Squared Values

#6 Post by Lina PhD student » 2020.02.28 14:34

Hi everyone,
Could you help me please understand how sigma(s), the experimental errors, are calculated in the Chi squared equation of CRYSOL, X2 = (1/N) * sum [ ( Iobs(s) - cIcalc(s) )2 / sigma(s)2 ] ?
Thanks in advance

User avatar
AL
Administrator
Posts: 707
Joined: 2007.08.03 18:55
Location: EMBL Hamburg, Germany
Contact:

Re: how sigma(s), the experimental errors, are calculated

#7 Post by AL » 2020.02.28 16:12

The experimental errors sigma(s) are taken from the input data file, third column.

Lina PhD student
Member
Posts: 5
Joined: 2020.02.28 11:46

Re: Chi-Squared Values

#8 Post by Lina PhD student » 2020.02.28 16:29

Which input data file ?
If I apply the following CRYSOL command "crysol structure.pdb target.dat" with structure.pdb is a random frame of my structure trajectory and target.dat contains deux columns (q, Intensity). In this case what is sigma(s) in the Chi squared value calculated by CRYSOL ?

Thanks in advance

User avatar
AL
Administrator
Posts: 707
Joined: 2007.08.03 18:55
Location: EMBL Hamburg, Germany
Contact:

Re: Chi-Squared Values

#9 Post by AL » 2020.03.02 18:04

Lina PhD student wrote:
2020.02.28 16:29
Which input data file ?
target.dat - it should contain the experimental SAXS data, three columns: q, I(q), sigma(q).
(q and s both mean the same).
If your SAXS data file does not have the third sigma column then CRYSOL will invent the experimental errors and compute the chi-squared using these.

Lina PhD student
Member
Posts: 5
Joined: 2020.02.28 11:46

Re: Chi-Squared Values

#10 Post by Lina PhD student » 2020.03.02 19:18

My target.dat doesn't have a third column.
Actually I am trying to reproduce CRYSOL calculation of the chi-squared value. Thus, I need to better understand how CRYSOL invents the experimental errors ?
Thanks in advance.

SaxsMax
Administrator
Posts: 353
Joined: 2007.08.03 19:16

Re: Chi-Squared Values

#11 Post by SaxsMax » 2020.03.03 14:28

It does polynomial smoothing of the non-negative data. If the smoothed array still has significant systematic
deviations, 3% relative error is assigned.
In practice, the third column in the resulting *.fit file is the used error value.

Lina PhD student
Member
Posts: 5
Joined: 2020.02.28 11:46

Re: Chi-Squared Values

#12 Post by Lina PhD student » 2020.03.04 17:27

I have my experimental data (actually it is not a real experimental data, it is generated by crysol to test the value of chi-2) and my structure pdb file. I try to apply chi-2 equation (mentioned in the article D. Svregun, C. Barberato, M. H. J. Koch) but it does not work. In the chi-squared equation, I use experimental data file as Ie in the chi-squared equation, I use intensity file of my pdb structure (obtained by crysol) as Icalculated in the chi-squred equation and I use 0.03*Ie=sigma to calculate sigma.
However my calculate for chi-squared is not the same with the chi-squared value obtained by crysol.
Could you help me to better understand where is the problem ?

Thanks in advance.

SaxsMax
Administrator
Posts: 353
Joined: 2007.08.03 19:16

Re: Chi-Squared Values

#13 Post by SaxsMax » 2020.03.05 09:57

Could you please attach the resulting fit file?

Lina PhD student
Member
Posts: 5
Joined: 2020.02.28 11:46

Re: Chi-Squared Values

#14 Post by Lina PhD student » 2020.03.05 12:16

Thanks in advance
Attachments
1KH6_crysol.dat
experimental data (not exactly experimental data, this file is obtained by crysol, it's corresponding to the frame 0 of the structure)
(5.08 KiB) Downloaded 5 times
model_1_crysol_v2.pdb
pdb structure file
(135.37 KiB) Downloaded 5 times
model_1_crysol_v200.fit
fit file result of the command (crysol model_1_crysol_v2.pdb 1KH6_crysol.dat)
(5.74 KiB) Downloaded 3 times
model_1_crysol_v200.log
log file result
(2.88 KiB) Downloaded 4 times

SaxsMax
Administrator
Posts: 353
Joined: 2007.08.03 19:16

Re: Chi-Squared Values

#15 Post by SaxsMax » 2020.03.05 14:57

I've used the crysol binary from Atsas-3.0.1 release and obtained the attached fit file.
The obtained chi^2 value is 3.355. The chi^2 computed from the fit file taking lines 3 to end is also 3.355.
Can you reproduce this result?
Attachments
model_1_crysol_v202.log
(81.76 KiB) Downloaded 5 times
model_1_crysol_v202.fit
(5.9 KiB) Downloaded 5 times

Post Reply