source: svn/trunk/paper/CommPhysComp/detailed_answers.txt@ 590

Please find below the answers to the referee's questions:
---------------------------------------------------------

> Reviewer #1: Dear authors,
>
> I have read your nice paper with great interest and I am
> mostly very happy with the content. I think the program 
> is very useful for quick but still realistic physics 
> estimates at the LHC and will hopefully be used  
> a lot by the theory (and experimental) community in the 
> next years.
>
> I have a number of comments that concern the text and 
> should be fairly quick to implement:
>
>
> Abstract: remove the bracket concerning Les Houches 
> and HepMC file format
done



> Section 1. Give citations to the conceptually similar 
> programs Atlfast and CMSJET from the ATLAS and CMS 
> collaboration. These are not freely available to people 
> outside the collaborations, but follow the same 
> concept and are established since a long time. There is 
> also the freely available, but much simpler AcerDET 
> program, that should perhaps be cited.

We have added AcerDET and PGS which are publicly available. As far as we know, CMSJet as been replaced by FAMOS and then by CMS FastSim, but are not publicly available. Same holds for ATLfast (I and II) in the ATLAS collaboration, so we prefer not to mention them



> Sec 2.2. When writing about tracks, readers might expect 
> to get the 5 helix parameters of tracks. Perhaps mention 
> directly that these are not available (especially no 
> impact parameters).
Insertion of the "while the helix parameters of tracks, especially the impact parameters are not." final sentence at the end of Sec 2.2



> Page 4, line 28: add "E in GeV" or similar to the formula
done




> Page 4, line 60: you deposit all energy of hadronic 
> particles in the hadronic calorimeter, which is a rather 
> crude approximation of a real calorimeter. If I 
> understand your concept correctly, you do this in 
> order to get the right amount of energy smearing for the 
> jet and MET resolutions. If this is the case, mention 
> that the longitudinal separation of the calorimeters is a 
> technical aid for this purpose and shouldn't be used in 
> analysis to mimic things like jet->electron fake  
> rates. Otherwise there is a risk that people will develop 
> analysis believing that this separation is based on real 
> physics, not simplifications in simulation.
Modified sentence: "To simplify the simulation, electrons and photons ..."



> Page 4, 2nd column, hight of line 17: "is F is" remove 
> first "is"
The non-needed "is" has been removed


> Sec 3.1. For photons you don't have a direction from 
> tracks and hence the position of the primary vertex is 
> important in the determination of the photon 3-vector. 
> Mention that diphoton resonances (H->gamma gamma)  
> also depend on this resolution, which is not simulated 
> (this is perhaps less important for ATLAS: they have the 
> photon pointing from the calo)
New sentence added: "As there is no associated track to photons, their reconstructed kinematics is less good than for electrons. Consequently, the mass resolution of di-photon resonnances is affected."



> Sec 3.1. In your electron simulation you take only the 
> energy from the electron itself, not from any physics 
> bremsstrahlung that is often collinear with the electron 
> (I mean the real emission bremststrahlung directly from 
> the Z->ee decay, not from the magnetic field in the ID).  
> This leads in Z->ee events to a visible distortion of the 
> line shape. I am aware that there is no easy solution to 
> this problem and just adding all photons in the same cell 
> leads to bias for others reasons. However,  
> the reader should be warned that the Z peak will be 
> slightly shifted due to the missing bremsstrahlungs 
> energy.
Modified paragraph: "Real electron ($e^\pm$) and photon candidates are identified if they fall into the acceptance of the tracking system and have a transverse momentum above some threshold (default: $p_T > 10~\textrm{GeV}/c$). \textit{Delphes} assumes a perfect algorithm for
clustering and for recovery of detector-induced Brehmstrahlung. Electron energy is smeared according to the resolution of the calorimetric cell where it points to, but independently from any other deposited energy in this cell. Electrons and photons may create a candidate in the jet
collection. The $(\eta,\phi)$ position at vertex comes from their corresponding track vertex. As there is no associated track to photons, their reconstructed kinematics is less good than for electrons. Consequently, the mass resolution of di-photon resonances is affected. Similarly to full simulations, physics Brehmstrahlung from vertex is not
taken into account in the electron reconstruction in \textit{Delphes}. Consequently, resonances like $Z\rightarrow e^+ e^-$ may have a bias in
their reconstructed peak.



> Page 5, line 15: If you reconstruct an electron, is this 
> correlated to the 90% efficiency of the electron track? 
> Whatever it is, mention it.
Sentence "Moreover, charged leptons are added in their respective collections i their associated track is available." has been added in Section 3.1.



> Page 5, line 22: Same for muons, is the 90% track 
> efficiency taken consistently into account?
Same as for electrons



> Page 5, line 59: because photons from physics 
> bremsstrahlung are not added to the electron, they appear 
> in the calo isolation energy. You should mention that 
> this can lead to unrealistically high isolation  
> energies in the case of very high energetic electrons 
> (while in a full simulation most brems photons overlap 
> with the electron and do not enter the isolation energy).
The following explaining sentence has been added in the end of the isolation section:  "Finally it should be mentioned that because photons from physics bremsstrahlung are not added to the electron, they appear in the calorimetric isolation energy. This can lead to unrealistically high
isolation energies in the case of very high energetic electrons while in a full simulation most bremsstrahlung photons overlap with the electron and do therefore not enter in the isolation energy."


> Sec 3.2. Perhaps mention that the jet energy scale is 
> "correct" by default, because all particles enter the 
> calorimeter with an average  
> response function of 1. However, out-of-cone corrections 
> (the dR size dependend) to the original parton energy are 
> not included and hence the reader can not expect to get 
> "correct" hadronic W or top masses (people from theory 
> might not be aware of this effect if they only know 
> partonic NLO final states).
Following sentence added:
"By construction, the jet energy scale is close to reality and should not be corrected. However, no corrections are applied if particles do not enter in the constituent list for the jet algorithm. This has an impact on the mass reconstruction of fully hadronic objects like $t$ quark or $W$ boson."



> Sec 3.2. Energy flow, first line: into->in
Modified



> Sec 3.2. Energy flow: clarify what is subtracted. The 
> true energy of the track or the smeared energy of the 
> particle that was put into the cell.
The sentence has been modified: "When using \textit{energy flow}, in \textit{Delphes}, the chosen jet algorithm is applied on the smeared energies of both the tracks and the modified calorimetric cells. To avoid double-counting, the energy in the calorimeter cells has been corrected by
subtracting the true track energy from the true cell energy, before their smearing."



> Sec. 3.3. You assume and apply a b-tagging efficiency of 
> 40%. Explain why line 13 says that the efficiency is 
> below 40% and what is the  relevance of secondary 
> vertices in this context, if they are not used after all?
The sentence was not relevant and is therefore removed since indeed no secondary vertex is applied.


> Page 6, line 18/19: full detector simulation ->  full 
> detector reconstruction. Tau identification is a 
> reconstruction issue.
simulation -> reconstruction


> Page 6, line 21/22: perhaps replace: associated -> in 
> combination with 
replaced


> Figure 4: figure and text do not match. The figure show 
> R^tracks, the text explains the dR of the jet algorithm. 
> These two R can be identical by coincidence, but don't 
> have to be.
The label of the figure has been modified accordingly


> Sec. 3.4. Purity: is the 66% relative to all hadronic tau 
> decays or only to 1-prong hadronic tau decays?
" relative to all hadronic $\tau$ decays" has been added at the end of the sentence.



> Sec. 3.4. general: it would be nice if the program could 
> contain in a future release the option to also tag 
> 3-prong tau decays. These are and will be used by the 
> experimental community and their fraction is not so  
> small that they can be neglected. A alternative tau 
> treatment with efficiencies and rejections as you have it 
> now for b-jets would be very nice.
Addition of the footnote: In release 1.9, the 3-prong tau decays are also included.


> Sec. 3.5. perhaps mention that calorimeter noise and the 
> degradations of the MET performance due to noise (mainly 
> outside jets) is not simulated.
Addition of the following sentence at the end of the section: "Moreover, the degradations of the missing transverse energy performance due to noise is not simulated."


> Sec. 5.2. you state that the roman pots at 220m detect 
> protons between 120 and 900GeV. When looking at the 
> table, this should be an energy loss of 120-900GeV?
Addition of the missing "loss" word: "For instance, roman pots at 220 m from the  IP and 2 mm from the beam will detect all forward protons with an energy loss between 120 and 900 GeV."


> Sec. 5.2. Does Hector run with the default LHC parameters?
Hector runs with the following parameters: 7 TeV/beam and beta*=0.5m for a collision mode.  The exhaustive list of Hector parameters are detailed in the user guide (as well as in the detector card) in section A.2.1.

> Sec. 5.2. particules -> particles
corrected


> Sec. 6 can you give the CPU time in standardized CPU 
> units? In 1-2 years from now the unit of a "regular" 
> laptop might no longer be well defined...
Added a footnote with CPU benchmark and a new bibliography entry.


> Page 8, last sentence: remove "simple": after isolation 
> also electrons/muons will no longer match the input 
> depending on the complexity of the event topology.
the "simple" word has been removed


> Fig. 8: is this with or without energy flow? Since you 
> have the option of both, would be nice to see both.
The text " and no energy flow correction" has been added to the captions of fig 8 and 9.  We did not produce these plots with Eflow as we do not have the corresponding  (publicly available) results from CMS and ATLAS.


> Sec. 6.1.: Can you give some quantitative numbers how 
> different/similar your jet resolution is compared to 
> ATLAS and CMS in the low and high pT regime? "good" 
> agreement is very subjective.
The numbers have been added in the text:
   - CMS:   $3.1~\%$ at $E_T^\textrm{MC}=50~\textrm{GeV}$ and $4.2~\%$ at
$500~\textrm{GeV}$.
   - ATLAS: $2.5~\%$ at $E^\textrm{MC}=50~\textrm{GeV}$ and $4.7\%$ at
$500~\textrm{GeV}$.


> Sec. 6.3. Since you have this nice tau 1-prong simulation 
> it would be very good to not only see how good 
> efficiencies match but also how well the qcd rejection 
> works. For b-jets you should get this quite well by  
> construction as long as you stay in the same topology. 
> However, for the taus this is not clear to me and what 
> counts after all is the pair of efficiency and rejection.
The following sentence has been added at the end of the section: "Using the same $pp \rightarrow ggX$ events as in section \ref{sec:jetresol} with a gluon transverse momentum at $80$ and $640~\textrm{GeV}/c$, the mis-identification rate of jets as $\tau\textrm{-jets}$ is evaluated at
$6.4\times 10^{-4}$ and $1.7\times 10^{-4}$ respectively."



> Sec. 8, 1st sentence: introduced -> intended?
corrected


> Sec. A.1. The address for the Delphes download seems to 
> be the user directory of S. Ovyn. Please move this to 
> some public address that can be expected to be available 
> for a long period of time, for example your hepforge area:
> http://www.hepforge.org/downloads/delphes
> User directories usually vanish once people move to other 
> universities.
The download place "http://projects.hepforge.org/delphes/files/Delphes" has been added





> Reviewer #2: Review: Delphes, a framework for fast 
> simulation of a generic collider experiment
>
> The presented document is a solid piece of work and 
> summarizes the purpose and mechanisms of Delphes to a 
> good level.
> However, there are several comments and key issues that I 
> would like to  
> address, which would help to clarify certain aspects of 
> this simulation  
> toolkit. Given that these comments are dealt with or 
> answered/modified  
> in a satisfactory way, this note qualifies for 
> publication.
>
> General Comments:
> ================
>
> The document is very imbalanced in terms of detector 
> subsystem: inner tracker and muon system aspects are very 
> sparse, while the calorimetric response is described to a 
> very detail (including a short discussion of  the 
> properties of certain jet algorithms that should not be 
> within the scope nor propose of this document, or only in 
> the appendix).

The jet algorithm explanations have been removed. We now only enumerate them. Their descriptions are although remaining in the appendix 



> No result concerning the TRACKER and MUON simulation in 
> Delphes is presented, the number of tracks inside a jet 
> cone is simply a geometrical measure that does no need a 
> TRACKER setup.
>
> The document describing Delphes, being focussed on 
> general-purpose experiments, would simply profit from a 
> more general-purpose view.  In this respect, I find the 
> description of the TRACKER response misleading/confusing:
> - p2.32l: "multiple scattering ... also neglected"

modified sentence: "The propagated particles do not suffer secondary interactions nor multiple scatterings that would affect their integrity or path throughout the detector. Photon conversion and bremsstrahlung due to the magnetic field are also neglected."

> - p2.15r: the resolution of tracking is mentioned 
> (intrinsic ? multiple scattering contribution ?)
> this is mentioned later in sec 2.2 (tracks) and 2.3 
> (calocells)  
This question is addressed in the answer to question p3.52l here below.

> - p3.Fig1: Smearing Tracks is mentioned in the figure, 
"Tracks are reconstructed" is mentioned in the label

> - p3.52l: "No smearing is currently applied to track 
> parameters."
> This needs additional clarification:
> - are the tracks smeared or not (if so, I guess only with 
> the intrinsic component and not the multiple scattering 
> contribution)?
"The momentum of tracks undergoes an intrinsic smearing, corresponding to the experiment capabilities for each particle. No further smearing is currently applied on track parameters."


> - "tracks are reconstructed" in label of Fig1 has to 
> disappear, because only a general tracking efficiency is 
> applied
replaced by "Tracks are identified".

> Multiple references:
> - throughout the text, references (such as [6][7]) are 
> multiply cited  
> (even in figure labels). Proposal: state clearly that the > ATLAS/CMS results for the following section have been 
> taken from [6][7] and leave it by such.
corrected


> Comments in detail:
> ================
>
> p1.
> - 59r: reference to Geant: the word "package" is pretty 
> much slang, I  
> would propose something like "toolkit" or similar.
corrected

> p2.
> -  6l: "developped" -> "developed"
spelling corrected

> -  9l: "reactions" does not sound very nice here, 
> probably "processes"
"Processes" has been written instead of "reactions"

> - 15l: "most crucial experimental features": this may be 
> a rather subjective judgment (see general comments), 
> studies involving/requiring TRACKING related quantities, 
> b-tagging, fake lepton backgrounds, etc.  
> may not share the opinion on this list. (I am aware that 
> this is a fast  simulation and it is not within the 
> purpose of such to generate fake signals)
modified text: "Delphes includes important experimental features"


> - 32l: "multiple scattering", see general comment on 
> TRACKING section
ok

> - 55r: is the global reconstruction efficiency applied 
> independently from the particle type ? above 1 GeV muons 
> tend to have a reconstruction efficiency close to 100% 
> for CMS/ATLAS, e.g. while pions/hadrons are when applying 
> some quality cuts more towards 80% for both experiments.  
> There seems to be no pseudo- rapidity dependence of the 
> reconstruction efficiency which is indeed the case in 
> more realistic detectors. If reconstruction efficiency is 
> assumed to be uniform and independent of the particle 
> type, this should be explicitly mentioned.
The sentence "This reconstruction efficiency is assumed to be uniform and independent of the particle type." has been added since indeed Delphes is not taken into account the different particle types

> - Fig2: the wireframe model is quite confusing, also it 
> hides a more important feature: the coverage cones close 
> to the beampipe. It is almost repeated in Fig. 11, but 
> with a cut-out view. I would keep only  
> one Figure.
This figure has been removed. It was replaced by the former figure 11 that has disappeared from Section 7.

> p3.
> - Fig1: figure and label, see general comments on 
> TRACKING section
Comma's have been added in the Figure since it describes the different running flow of the Delphes code: first the tracks are found (for charged particles) taking into account their bending due to the magnetic field.
This step is followed by the smearing step,...

> - 52l: see general comments on TRACKING section
ok

> - 53l: does Delphes allow for shifted primary vertex 
> position: if so (\eta,\phi) at the vertex should be 
> directional parameters, while (\eta,\phi)_{calo} should 
> be parameters of the intersection point.  
> hence, \eta_{vertex} =/= \eta_{calo} both in value but 
> also conceptually. (particles from secondary vertices 
> suffer from this anyway).
Delphes is not meant for shifted primary vertex position. However, the correction can be computed. The \eta_calo is the coordinate where the particle hits the calorimeter in
the detector frame. To get the real \eta for a displaced (i.e. shifted primary- or secondary-) vertex, one should correct this quantity from the track origin and the
observed \eta_calo. We have not added any remark in the text.


> p4.
> - Fig3: this figure does not contain a lot of 
> information, I would propose a table instead to give an 
> idea about the cell sizes.
The requested numbers are available in the appendix, in the description of the detector parameter cards. Unfortunately, such a table would be long and difficult to read. For clarity, a reference to the appendix has been added.


> - 7r: K_s and \Lambda, ... decays are not performed, but 
> the energy is deposited as a fraction in the calorimeter. > Where ? Is the transport done in direction of the mother 
> particle ?
Addition of the following sentence: "The position of these deposits corresponds to the calorimeter entry point of the mother particle."

> - 8-14r: many lines for a simple concept, would shorten 
> this
done

> - 55r: "some additional reconstruction cuts" is vague
The end of the sentence: "and some additional reconstruction cuts." has been removed. It is not necessary anymore since the cuts on electrons are detailed in the appendix. Moreover, the isolation is explained in a following sub-section

> - 58r: "However, when needed ... " I would personally 
> delete this sentence, telling the user he could do the 
> work on  his own.
The sentence has been deleted!

> p5.
> - 7l: why are electrons but in particular photons in 
> reconstruction restricted to the tracking system 
> acceptance ?
The sentence has been modified to: "Real electron ($e^\pm$) and photon candidates are* identified* if they fall into the acceptance of the tracking system and have a transverse momentum above some threshold (default: $p_T > 10~\textrm{GeV}/c$). " Indeed if they fall out of the
tracker coverage, the particles are detected but not identified. Delphes does this since the detection of the track is mandatory in order to differentiate a photon and
an electron.

> - 40-42r: I would move the discussion of the different 
> jet algorithms (if any) entirely to the appendix (p19. 
> 45+), it is not the scope of the document to do so.
Already removed due to the comment of the other referee

> p6.
> - 11l: "Therefore, ... " - I don't understand the 
> consequential reasoning, would delete "Therefore, ..."
The "Therefore" word has been removed => In current version....

> p7.
> - 2-14r: A long paragraph describing how it is not done 
> ... distracts  
> the reader, better focus on how it is done
The paragraph has been shortened. It is now: "A trigger emulation is included in \textit{Delphes}, using a fully parametrisable \textit{trigger table}  \citep{qr:triggercard}. While triggers in real experiments are intrinsically based on reconstructed data with a worse resolution than final analysis information, in \textit{Delphes} the same information is used for the trigger emulation and for final analyses."

> p8.
> - 43r: "a regular laptop" - this is not a standardized 
> system, better use normalized CPU seconds or similar
footnote added:
\footnote{Performances obtained on an Intel(r) Pentium M processor ($1.73$ GHz), $1$ GB RAM, Chipset $915$GM. This processor scores $450$ on CpuMark benchmark from PassMark 2010 (c)~\citep{bib:cpumark}. In case of large amount of events, the duration of read/write operations on hard disk is sensible (about 6\% of the time for the $4400$ rpm hard disk of the precited computer) and might improve with fast storing devices.}


> - 48r: a bit simplified view electron/muon resolutions 
> are of course non-gaussian and usually dependent of pT/eta
The pT/eta dependence could be added in a later version of Delphes but is not available in versions 1.8 and 1.9.


> p9.
> - Fig8: bottom line of the axis box seems to be missing 
> (at least in my print-out)
It seems to be OK in the pdf paper and in the print test we did...

> - Fig8/Fig9: include the markers into the label
The "stars" have been added in the four validation graphs

> p10.
> - 1l: "is negligible in the studied sample" - this is 
> true, but one could mention that for e. g. W->\mu \nu 
> muons should be corrected for
The sentence "Indeed for e.g.$W\rightarrow \mu \nu$ samples the missing transverse energy should be corrected for the presence of muons." has been added. This correction has been validated in my PhD Thesis by an accurate comparison with CMSSW http://cp3.phys.ucl.ac.be/upload/theses/phd/ovyn.pdf by comparing Table 5.8 and Table 4.7)

> p11.
> - 22l: "a simple mouse action" - sounds a bit clumsy
corrected

> - Fig12/13: I suggest only keeping one of the pictures; 
> it should be clear to the reader that different event 
> topologies have different characteristics (also in the 
> visulatisation), the purpose of your document must be 
> showing the visualisation tool.
The second Figure (Fig 13) has been removed.

> Appendix:
>
> p13.
> - 9l: "its sources" probably change it into "source code"
done

> p16.
> - 8l : do you want latex "\textit{Hector}" comments in the user manual ?
\textit{} removed!

> p21.
> - 20l: "For more information, refer to ROOT 
> documentation." I would personally shorten this paragraph 
> (the ROOT description), and refer to the root 
> documentation by citation.
We prefer to keep this section with the description since several users of Delphes encountered problems with this...