Section 6

PAGE 12:

Section 6.1

Last line: What does ”alternatively” mean ? If one choice is for electrons and the other for muons, the authors should state it clearly.

the last sentence has been changed to make the statement more clear.

Footnote: this statement deserves a complete section, with the list of parameters used in the default CMS and ATLAS configurations, as well as possible explanations as to why this parameter choice was made, and a comparison with the actual CMS and ATLAS resolutions and granularities.

The authors disagree with this comment. The reasons are:

• the resolutions in the Delphes CMS and ATLAS cards are directly taken from CMS and ATLAS.
• the resolutions from CMS and ATLAS are taken directly from the cited papers, and it would be redundant to quote there here.
• the only possible difference is in the calorimeters granularity. Both the CMS and ATLAS configurations in Delphes use the granularity of the HCAL detector. As said in the calorimeter section, the ECAL granularity is exactly the same as the HCAL >>> granularity in Delphes. This comment has now been added in the "jet and met"validation sub-section". A table with the actual HCAL granularity of the LHC experiments (already public in the relevant technical design reports of CMS and ATLAS) >>> would be of poor interest to the reader and redundant.

PAGE 13

Figure 3

• It is not clear what the grey bands are in this plot. Shouldn’t they be removed? In CMS, they are supposed to cover differences between the simulation and the data in CMS, not the difference between Delphes and CMS. This comment is valid for all plots. Strangely enough, the ATLAS fred band width is way smaller than that for CMS. Does it represent the same thing ?

the grey bands mean indeed different things in the left and right plots. The caption has been extended in order to explain the details required by the referee.

• For all plots, it is important to have the statistical uncertainty bars indicated, or to state that they are covered by the size of the markers. In the latter case, an explanation is needed for the apparent scatter of the DELPHES points, and to compare this scatter with the input resolution function.

The apparent scatter is just due to the fact the the parametrisation of the resolution is binned, and has been chosen to match approximately that of CMS and ATLAS. The choice was made to adopt round values which may result in the apparent scatter. As these plots are just an illustration of a parametrisation which is correct by construction, we believe that no further explanation is needed.

• For all plots, label, legends, etc... are way too small to be readable.

this comment has been addressed for all the plots.

Figure 4

Looking at Ref. [20], and in particular its slide 33, my impression is that the DELPHES resolution is a factor 2 optimistic with respect to the CMS resolution. It seems that DELPHES parametrized the Gaussian width of the core of the CMS resolution, rather than the effective 68% width.

the referee is correct, the electron resolution was parametrized with the Gaussian width, since the resolution in Delphes is Gaussian by construction. By proceeding this way we are voluntarily neglecting the tails effects in Delphes.

PAGE 14

Figure 5

On the CMS side, it would be nice to have an explanation for the following effects (already alluded to above):

• why does the calo jet resolution curve saturates at low jet pT ?
• why does the PF jet resolution curve of Delphes departs from CMS at high pT? (it’s probably because the tracker pT resolution is used to determine the PF track pT) And at low pT ? Actually, the DELPHES jet pT resolution is almost independent with pT: it goes from 10 to 7% when varying the pt from 30 to 500 GeV/c, while the actual CMS PF varies from 14% to 5%.
• why does the PF curve show a discontinuity between the 1st and 2nd points ?

This plot has been re-done. The HCAL calorimeter resolution has been set to the actual CMS resolution, and the energy-flow implementation is as explained in Section 2.3. As a result we have a perfect agreement at all medium and high pt values. At low pt there is a discrepancy for 20<pt<30 which is not understood. However we believe this discrepancy to have very low impact on physics analyses, that most often consider jets with pt > 30. The small discrepancy observed in the 30 > pt > 40 GeV bin is ~1% in resolution. This comments have been added in the text.

Once the effects are understood, it would be important to fix the imple- mentation in DELPHES. These differences are important for data analysis, and may to DELPHES user draw wrong conclusions from his DELPHES studies (of particular importance if DELPHES is used to define the upgrade strategy of the expensive LHC detectors).

addressed. However, the referee, as well as the LHC experiments, should be aware that any study performed with Delphes should be understood as preliminary. A Delphes based study should be perfomed after a pure parton-level and before a geant based fast or full-simulation study. As a result, the authors are perfectly happy with an agreement with a few percents discrepancy in the physics object resolutions.

Section 6.3

Par 2:

It is not clear if the CMS study is made with of without pile-up. As pile-up and its particle-flow mitigation are an important adds-on to DELPHES 3.0, it would be nice to have an illustration of their performance here.

addressed. Added a sentence at the beginning of the paragraph stating that only the fake MET is done with pile-up.

what do we say here? (michele suggested answer)

We don't want to create to much imbalance between CMS and ATLAS (the only exception is electrons, but we did not find the relevant plot for ATLAS), so we have decided to produce a real MET validation plot for CMS and and the fake validation for ATLAS. This choice was simply driven by the fact that the real MET validation plot was not found in the ATLAS. In addition, pile-up mitigation for MET has not been addressed in this note, since it is not done. As a matter of fact PU mitigation on the MET relies on complex multi-variate algorithms in CMS which are out of scope in Delphes. The only advantage in using the energy-flow particle instead of calorimeter tower for computing MET in the presence of pile-up is the superior resolution of track in comparison to towers. PU tracks cannot simply be removed (in contrast to what is done for jets).

PAGE 16

Par 1:

L3: The anti-kT algorithm has no ”cone” - and ”a cone R = 0.5” has no meaning even for a cone algorithm. (Note: the same mistake occurs in Section 7.2)

L9/11: The slight difference of efficiency is a large difference (20%), which might the DELPHES user draw incorrect conclusions from the abilities of his/her analysis. The explanation given here should be checked, and the cul- prit (jet energy correction or b tagging efficiency should be fixed in DELPHES.

Par 2:

Bullet 1: ”any parton from the top quark decay” ! ”any parton from the decay of either of top quarks”. I find the definition of unmatched rather un- natural. If the three jets from the ”hadronically-decaying top” were matched, I fail to understand why the event is classified as unmatched, even if the other b for the other top is unmatched.

Par 3:

L5: Why are the distributions not normalized to the number of events ? Is it because the number of permutations/event is vastly different in DELPHES and in CMS ? If it is the case, shouldn’t DELPHES be fixed ?

L9/10: ”Pile-up, not considered in the present study, can degrade the jet energy resolution.” : A strong emphasis is put on the ability of Delphes 3.0 to simulate pile up, and to simulate its mitigation procedures based on the PF reconstruction. Since Delphes is fast, I would assume that it would take no time to redo this study with pile-up. It is disappointing for the reader not to see this validation in the paper. It actually casts doubts on the DELPHES ability to accurately simulate pile-up and its mitigation, which is surely not what the authors aim at.

PAGE 17

Figure 7

The bottom inserts of all plots are difficult to understand. The label ”rel. diff.” makes the reader guess that they show the relative difference (i.e., the ratio - 1) of the two distributions, but a quick look at the distributions leads the reader to doubt about it. For exampe, the right plot shows a 20% difference between the two distributions around the maximum, which is not visible in the ”rel. diff.” plot. Maybe the wrong scale was chosen for the bottom inserts ?

PAGE 18

Par 4: The reader is again disappointed to see that, in the search for VBF- produced Higgs boson with pile-up, for which it is said that pile-up has a pretty large and negative impact, the authors decided to use calorimeter jets instead of particle-flow reconstruction, aimed exactly at mitigating pile-up effects. Again, it casts doubts on the ability of DELPHES to simulate pile-up in particle reconstruction, and to simulate its mitigation with particle-flow reconstruction. The paper has therefore the effect opposite to what the authors are aiming at.

Criterion 2.

The pT cut used to count light jets between j1 and j2 ought to be given. Are these four cuts used in the CMS analysis which the authors are using for comparison ?

PAGE 20

Par 2:

L4: ”accurate productions in high pile-up scenarios should solely rely on full simulation tools” is very strong, and probably incorrect statement, and should probably be removed (or seriously rephrased).

First, it casts again large doubts on the pertaining DELPHES ability to simulate pile up interactions.

Second, the fact that, maybe, DELPHES cannot deal with high PU en- vironment does not mean that other fast simulation tools cannot do. For example, it seems the CMS fast simulation deals pretty well with the high pile-up produced by LHC in 2012.