Changes between Version 1 and Version 2 of Private/RefereeComments/Section_6

Timestamp:

Aug 26, 2013, 9:01:13 AM (11 years ago)

Author:

cp3-support

Comment:

--

Private/RefereeComments/Section_6

@@ -1 +1 @@
 = Section 6 =
 
+PAGE 12:
+
+Section 6.1
+Last line: What does ”alternatively” mean ? If one choice is for electrons
+and the other for muons, the authors should state it clearly.
+
+Footnote: this statement deserves a complete section, with the list of
+parameters used in the default CMS and ATLAS configurations, as well
+as possible explanations as to why this parameter choice was made, and a
+comparison with the actual CMS and ATLAS resolutions and granularities.
+
+PAGE 13
+
+Figure 3
+
+ * It is not clear what the grey bands are in this plot. Shouldn’t they be
+removed? In CMS, they are supposed to cover differences between the
+simulation and the data in CMS, not the difference between Delphes
+and CMS. This comment is valid for all plots
+Strangely enough, the ATLAS fred band width is way smaller than that
+for CMS. Does it represent the same thing ?
+ * For all plots, it is important to have the statistical uncertainty bars
+indicated, or to state that they are covered by the size of the markers.
+In the latter case, an explanation is needed for the apparent scatter
+of the DELPHES points, and to compare this scatter with the input
+resolution function.
+ * For all plots, label, legends, etc... are way too small to be readable.
+
+Figure 4
+
+Looking at Ref. [20], and in particular its slide 33, my impression is that
+the DELPHES resolution is a factor 2 optimistic with respect to the CMS
+resolution. It seems that DELPHES parametrized the Gaussian width of the
+core of the CMS resolution, rather than the effective 68% width.
+
+PAGE 14
+
+Figure 5
+
+On the CMS side, it would be nice to have an explanation for the following
+effects (already alluded to above):
+ * why does the calo jet resolution curve saturates at low jet pT ?
+ * why does the PF jet resolution curve of Delphes departs from CMS at high pT? (it’s probably because the tracker pT resolution is used to
+determine the PF track pT) And at low pT ? Actually, the DELPHES jet pT resolution is almost independent with pT: it goes from 10 to 7%
+when varying the pt from 30 to 500 GeV/c, while the actual CMS PF varies from 14% to 5%.
+ * why does the PF curve show a discontinuity between the 1st and 2nd points ?
+
+Once the effects are understood, it would be important to fix the imple-
+mentation in DELPHES. These differences are important for data analysis,
+and may to DELPHES user draw wrong conclusions from his DELPHES
+studies (of particular importance if DELPHES is used to define the upgrade
+strategy of the expensive LHC detectors).
+
+Section 6.3
+
+Par 2:
+
+It is not clear if the CMS study is made with of without pile-up. As pile-up
+and its particle-flow mitigation are an important adds-on to DELPHES 3.0,
+it would be nice to have an illustration of their performance here.
+
+PAGE 16
+
+Par 1:
+
+L3: The anti-kT algorithm has no ”cone” - and ”a cone R = 0.5” has
+no meaning even for a cone algorithm. (Note: the same mistake occurs in
+Section 7.2)
+
+L9/11: The slight difference of efficiency is a large difference (20%), which
+might the DELPHES user draw incorrect conclusions from the abilities of
+his/her analysis. The explanation given here should be checked, and the cul-
+prit (jet energy correction or b tagging efficiency should be fixed in DELPHES.
+
+Par 2:
+
+Bullet 1: ”any parton from the top quark decay” ! ”any parton from the
+decay of either of top quarks”. I find the definition of unmatched rather un-
+natural. If the three jets from the ”hadronically-decaying top” were matched,
+I fail to understand why the event is classified as unmatched, even if the other
+b for the other top is unmatched.
+
+Par 3:
+
+L5: Why are the distributions not normalized to the number of events ? Is
+it because the number of permutations/event is vastly different in DELPHES
+and in CMS ? If it is the case, shouldn’t DELPHES be fixed ?
+
+L9/10: ”Pile-up, not considered in the present study, can degrade the jet
+energy resolution.” : A strong emphasis is put on the ability of Delphes 3.0
+to simulate pile up, and to simulate its mitigation procedures based on the
+PF reconstruction. Since Delphes is fast, I would assume that it would take
+no time to redo this study with pile-up. It is disappointing for the reader not
+to see this validation in the paper. It actually casts doubts on the DELPHES
+ability to accurately simulate pile-up and its mitigation, which is surely not
+what the authors aim at.
+
+PAGE 17
+
+Figure 7
+
+The bottom inserts of all plots are difficult to understand. The label ”rel.
+diff.” makes the reader guess that they show the relative difference (i.e.,
+the ratio - 1) of the two distributions, but a quick look at the distributions
+leads the reader to doubt about it. For exampe, the right plot shows a 20%
+difference between the two distributions around the maximum, which is not
+visible in the ”rel. diff.” plot. Maybe the wrong scale was chosen for the
+bottom inserts ?
+
+PAGE 18
+
+Par 4: The reader is again disappointed to see that, in the search for VBF-
+produced Higgs boson with pile-up, for which it is said that pile-up has a
+pretty large and negative impact, the authors decided to use calorimeter jets
+instead of particle-flow reconstruction, aimed exactly at mitigating pile-up
+effects. Again, it casts doubts on the ability of DELPHES to simulate pile-up
+in particle reconstruction, and to simulate its mitigation with particle-flow
+reconstruction. The paper has therefore the effect opposite to what the
+authors are aiming at.
+
+Criterion 2.
+
+The pT cut used to count light jets between j1 and j2 ought to be given.
+Are these four cuts used in the CMS analysis which the authors are using for
+comparison ?
+
+PAGE 20
+
+Par 2:
+
+L4: ”accurate productions in high pile-up scenarios should solely rely on
+full simulation tools” is very strong, and probably incorrect statement, and
+should probably be removed (or seriously rephrased).
+
+First, it casts again large doubts on the pertaining DELPHES ability to
+simulate pile up interactions.
+
+Second, the fact that, maybe, DELPHES cannot deal with high PU en-
+vironment does not mean that other fast simulation tools cannot do. For
+example, it seems the CMS fast simulation deals pretty well with the high
+pile-up produced by LHC in 2012.
+