Section 6

PAGE 12:

Section 6.1

Last line: What does ”alternatively” mean ? If one choice is for electrons and the other for muons, the authors should state it clearly.

the last sentence has been changed to make the statement more clear.

Footnote: this statement deserves a complete section, with the list of parameters used in the default CMS and ATLAS configurations, as well as possible explanations as to why this parameter choice was made, and a comparison with the actual CMS and ATLAS resolutions and granularities.

The authors disagree with this comment. The reasons are:

(CD) We disagree...

• the resolutions in the Delphes CMS and ATLAS cards are taken directly from the cited papers, and it would be redundant to quote them here.
• the only possible difference is in the calorimeters granularity. Both the CMS and ATLAS configurations in Delphes use the granularity of the HCAL detector. As said in the calorimeter section, the ECAL granularity is exactly the same as the HCAL granularity in Delphes. This comment has now been added. A table with the actual HCAL granularity of the LHC experiments (already public in the relevant technical design reports of CMS and ATLAS) would be of poor interest to the reader and redundant.

(CD) I think that the text should state the unique version number used for the paper. ATLAS and CMS cards may evolve in the future, and it should be clear to what minor version we refer in the paper.

PAGE 13

Figure 3

• It is not clear what the grey bands are in this plot. Shouldn’t they be removed? In CMS, they are supposed to cover differences between the simulation and the data in CMS, not the difference between Delphes and CMS. This comment is valid for all plots. Strangely enough, the ATLAS fred band width is way smaller than that for CMS. Does it represent the same thing ?

the grey bands mean different things in the left and right plots. The caption has been extended in order to explain the details required by the referee.

(CD) it should be said in the caption that the error bars are smaller than the dots. Avoid also to refer to colors.

• For all plots, it is important to have the statistical uncertainty bars indicated, or to state that they are covered by the size of the markers. In the latter case, an explanation is needed for the apparent scatter of the DELPHES points, and to compare this scatter with the input resolution function.

The apparent scatter is just due to the fact the the parametrisation of the resolution is binned, and has been chosen to match approximately that of CMS and ATLAS. The choice was made to adopt round values which may result in the apparent scatter. As these plots are just an illustration of a parametrisation which is correct by construction, we believe that no further explanation is needed.

• For all plots, label, legends, etc... are way too small to be readable.

this comment has been addressed for all the plots.

Figure 4

Looking at Ref. [20], and in particular its slide 33, my impression is that the DELPHES resolution is a factor 2 optimistic with respect to the CMS resolution. It seems that DELPHES parametrized the Gaussian width of the core of the CMS resolution, rather than the effective 68% width.

the referee is correct, the electron resolution was parametrized with the Gaussian width, since the resolution in Delphes is Gaussian by construction. By proceeding this way we are voluntarily neglecting the tails effects in Delphes.

(CD) I think that it should be said somewhere in the paper.

PAGE 14

Figure 5

On the CMS side, it would be nice to have an explanation for the following effects (already alluded to above):

• why does the calo jet resolution curve saturates at low jet pT ?
• why does the PF jet resolution curve of Delphes departs from CMS at high pT? (it’s probably because the tracker pT resolution is used to determine the PF track pT) And at low pT ? Actually, the DELPHES jet pT resolution is almost independent with pT: it goes from 10 to 7% when varying the pt from 30 to 500 GeV/c, while the actual CMS PF varies from 14% to 5%.
• why does the PF curve show a discontinuity between the 1st and 2nd points ?

This plot has been re-done. The HCAL calorimeter resolution has been set to the actual CMS resolution, and the energy-flow implementation is as explained in Section 2.3. As a result we have a perfect agreement at all medium and high pt values. At low pt there is a discrepancy for 20<pt<30 which is not understood. However we believe this discrepancy to have very low impact on physics analyses, that most often consider jets with pt > 30. The small discrepancy observed in the 30 > pt > 40 GeV bin is ~1% in resolution. These comments have been added in the text.

Once the effects are understood, it would be important to fix the imple- mentation in DELPHES. These differences are important for data analysis, and may to DELPHES user draw wrong conclusions from his DELPHES studies (of particular importance if DELPHES is used to define the upgrade strategy of the expensive LHC detectors).

addressed. However, the referee, as well as the LHC experiments, should be aware that any study performed with Delphes should be understood as preliminary. A Delphes based study should be perfomed after a pure parton-level and before a geant based fast or full-simulation study. As a result, the authors are perfectly happy with an agreement with a few percents discrepancy in the physics object resolutions.

(CD) I would add that "we made this point clear when consulted by the LHC collaborations. Delphes is designed as a

pheno tool, not as a replacement of fast simulation tools from the collaborations.

Section 6.3

Par 2:

It is not clear if the CMS study is made with of without pile-up. As pile-up and its particle-flow mitigation are an important adds-on to DELPHES 3.0, it would be nice to have an illustration of their performance here.

addressed. Added a sentence at the beginning of the paragraph stating that only the fake MET is done with pile-up. what do we say here? (michele suggested answer) We don't want to create to much imbalance between CMS and ATLAS (the only exception is electrons, but we did not find the relevant plot for ATLAS), so we have decided to produce a real MET validation plot for CMS and the fake validation for ATLAS. This choice was simply driven by the fact that the real MET validation plot was not found in the ATLAS. In addition, pile-up mitigation for MET has not been addressed in this note, since it is not done. As a matter of fact PU mitigation on the MET relies on complex multi-variate algorithms in CMS which are out of scope in Delphes. The only advantage in using the energy-flow particle instead of calorimeter tower for computing MET in the presence of pile-up is the superior resolution of track in comparison to towers. PU tracks cannot simply be removed (in contrast to what is done for jets).

PAGE 16

Par 1:

L3: The anti-kT algorithm has no ”cone” - and ”a cone R = 0.5” has no meaning even for a cone algorithm. (Note: the same mistake occurs in Section 7.2)

"cone" changed to "parameter"

(CD) probably not something to say to the referee, but while I fully agree for Kt, anti-Kt jets are still contained in a cone of radius R.

L9/11: The slight difference of efficiency is a large difference (20%), which might the DELPHES user draw incorrect conclusions from the abilities of his/her analysis. The explanation given here should be checked, and the cul- prit (jet energy correction or b tagging efficiency should be fixed in DELPHES.

We totally disagree with the referee here. The examples given here a purely illustrative and it is out of scope to fine-tune them. The purpose was precisely to show the opposite: without particular fine-tuning Delphes gives very reasonable agreement with the CMS analysis. We insist that a 20% difference is an acceptable difference since very often results (rates, efficiencies) obtained with full geant based simulation give larger discrepancies than 20% with respect to data.

(CD) Add: Most LHC analysis nowadays make use of signal-free regions to normalize backgrounds, and scale factors in excess of 1.2 are not unusual.

As a side comment, the efficiency was re-computed after the change in the energy-flow algorithm and the result was found to be the same.

(CD) More generally, private communications from CMS collaborations recently confirmed that an excellent agreement is observed when comparing Delphes to internal studies made with full simulation. Unfortunately, these are not results that we are allowed to show.

Par 2:

Bullet 1: ”any parton from the top quark decay” ! ”any parton from the decay of either of top quarks”.

addressed

I find the definition of unmatched rather un- natural. If the three jets from the ”hadronically-decaying top” were matched, I fail to understand why the event is classified as unmatched, even if the other b for the other top is unmatched.

the categories definitions were chosen in the cited CMS paper. We agree with the referee, however, for the sake of comparing to the CMS results the Delphes authors have to adopt the same definitions used in the CMS paper.

Par 3:

L5: Why are the distributions not normalized to the number of events ? Is it because the number of permutations/event is vastly different in DELPHES and in CMS ? If it is the case, shouldn’t DELPHES be fixed ?

the Delphes distributions are normalized to the total number of events in CMS (to account for the 20% in the selection efficiency). The total number of permutation is proportional to the event yield. The purpose of this example is indeed to show, as Table 1 illustrates, that we get the correct fraction of each permutation category in Delphes.

L9/10: ”Pile-up, not considered in the present study, can degrade the jet energy resolution.” : A strong emphasis is put on the ability of Delphes 3.0 to simulate pile up, and to simulate its mitigation procedures based on the PF reconstruction. Since Delphes is fast, I would assume that it would take no time to redo this study with pile-up. It is disappointing for the reader not to see this validation in the paper. It actually casts doubts on the DELPHES ability to accurately simulate pile-up and its mitigation, which is surely not what the authors aim at.

the whole analysis has been re-done with the new implementation of the energy-flow algorithm. The discrepancy in the top mass resolution is not present anymore, the reason being that the resolution of the new energy-flow jets matches almost exaclty that of the CMS particle-flow jets at the energy of interest ( 60 GeV ). We attribute therefore the small discrepancy that was previously observed to the imperfect implementation of the previous particle-flow algorithm rather than to the absence of pile-up simulation. The whole sentence mentioning the discrepancy has been dropped, since the discrepancy is not present anymore.

PAGE 17

Figure 7

The bottom inserts of all plots are difficult to understand. The label ”rel. diff.” makes the reader guess that they show the relative difference (i.e., the ratio - 1) of the two distributions, but a quick look at the distributions leads the reader to doubt about it. For exampe, the right plot shows a 20% difference between the two distributions around the maximum, which is not visible in the ”rel. diff.” plot. Maybe the wrong scale was chosen for the bottom inserts ?

The range has been changed to (-0.5,0.5) for "y" axis. The 10% (rather than 20%) difference the referee refers to is now visible.

PAGE 18

Par 4: The reader is again disappointed to see that, in the search for VBF- produced Higgs boson with pile-up, for which it is said that pile-up has a pretty large and negative impact, the authors decided to use calorimeter jets instead of particle-flow reconstruction, aimed exactly at mitigating pile-up effects. Again, it casts doubts on the ability of DELPHES to simulate pile-up in particle reconstruction, and to simulate its mitigation with particle-flow reconstruction. The paper has therefore the effect opposite to what the authors are aiming at.

We have followed the referee's suggestion and re-done the study considering energy-flow jets and energy-flow based pu mitigation.

Criterion 2.

The pT cut used to count light jets between j1 and j2 ought to be given.

It is said in the text that the requirement refers to the two leading jets non b-tagged jets. The pt threshold are defined by requirement 1.

Are these four cuts used in the CMS analysis which the authors are using for comparison ?

These cuts a probably similar in any VBF analysis performed in ATLAS or CMS. However, the goal here is not to compare to any existing analysis, but rather to give a simple example of utilisation of Delphes.

PAGE 20

Par 2:

L4: ”accurate productions in high pile-up scenarios should solely rely on full simulation tools” is very strong, and probably incorrect statement, and should probably be removed (or seriously rephrased).

First, it casts again large doubts on the pertaining DELPHES ability to simulate pile up interactions.

Second, the fact that, maybe, DELPHES cannot deal with high PU en- vironment does not mean that other fast simulation tools cannot do. For example, it seems the CMS fast simulation deals pretty well with the high pile-up produced by LHC in 2012.

"high pile-up scenarios" has been replaced by "extreme pile-up scenarios", which is we actually meant. To our knowledge there is no evidence that fast-simulation can cope to >100 simultaneous interactions environments, simply since these did not occur in any hadron collider yet.

(CD) I would reformulate further the sentence in the paper, saying that Delphes has not yet been compared to fullsim at extreme PU. Indeed, once done, it may become more quantitative in that region too. Still, by the time we reach such pileup conditions, experimental collaborations may find ways to cope with pileup that are not foreseen in Delphes.