[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: AD review of: draft-ietf-tewg-measure-05.txt
WG,
I'd like to hear/read more comments/feedback on my review.
Inline responses to the comments from Jerry
Thanks,
Bert
> -----Original Message-----
> From: Ash, Gerald R (Jerry), ALABS [mailto:gash@att.com]
> Sent: vrijdag 25 april 2003 23:43
> To: Wijnen, Bert (Bert); Tewg (E-mail)
> Cc: Ash, Gerald R (Jerry), ALABS
> Subject: RE: AD review of: draft-ietf-tewg-measure-05.txt
>
>
> Bert,
>
> Let's not throw the baby out with the bath-water. The I-D
> serves a critical need and should go forward.
>
Can you formulate the "critical need" that is being served?
> Regarding your comment:
> 'I would expect a CRISP set of "requirements for additional
> measurements, configurable/negotiable parameters/controls"
> ... but not the extensive exploration and text that I now see'.
> The requirements *are* there for additional measurements,
> configurable/negotiable parameters/controls, but they need to
> be made more prominent and crisp (as you say).
>
Well cedrtainly I could not see them... they are too well hidden
I guess :-(. So making them CRISP is certainly a good thing.
> I'd suggest to focus the main body of the I-D on these
> requirements/recommendations, and move the descriptive
> material to Annex(es). The text has evolved over a long time
> with many comments and responses, as such it has gathered
> some moss along the way... But the critical essence is
> there, and the requirements *need* to move forward to provide
> essential TE measurements.
>
So.... it seems we agree that this title:
A Framework for Internet Traffic Engineering Measurement
is not good and should be something aka
Requirements for essential TE measurements, parameters and controls
and then of course the main body of the document should focus on
that and be clear as to waht is a real requirement and why.
> Regarding your comment:
> 'if we were not yet doing any of it... then I wonder if this
> would be helpful at all.'
> There are many important requirements/recommendations in the
> I-D. As one example, the ability to collect traffic data to
> generate a traffic matrix (e.g., 'node-pair data', see Table
> in Section 10.1) is woefully lacking. To get a flavor of the
> problem/need, see these papers on the topic (links to papers
> at http://www.research.att.com/~jrex/papers.html#tmeas):
> A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford,
> G. True, "Deriving traffic demands for operational IP
> networks: Methodology and experience," IEEE/ACM Transactions
> on Networking, June 2001, pp. 265-279. (An earlier version
> appeared in Proc. ACM SIGCOMM, August/September 2000.)
>
I'd like to see in the document something aka:
REQUIREMENT: ability to collect traffic data to generate trafix matrices
DESCRIPTION: describe the details as to what traffic data is needed
JUSTIFICATION: describe what the problem is that needs to be solved
PRIORITY: x
Don't need to dwell too much on that... but have a concise and crisp
writeup. Maybe RFC3216 can serve as a good example.
> This work shows the difficulty in deriving a traffic matrix
> with today's measurements. A traffic matrix is basic to any
> network engineering or planning task.
>
> Other critical traffic/performance measurement requirements
> identified in the I-D are also lacking and need to go forward.
>
> A few more comments below.
>
> Jerry
>
> > - a lot of text... but not very focused and to the point
> (or at least I have trouble seeing the main points)
> > - not a "framework", rather an exploration of TE measurement
> > related topics (more like a summary-introduction)
>
> As above, the I-D serves a critical need. The text can be
> focused on the requirements/recommendations, as suggested,
> and the descriptive stuff moved to an Annex.
>
One could wonder how much of the descriptive stuff is really
helpful or usefull. But if it is in an ANEX, then maybe that is
OK. I would say however, that if we have the requirements
crisply formulated, with description, justification and priority,
then one would hope that a lot of additional text is not needed
(not even in an anex). Think about the audience. It would be
protocol developers and/or vendors who have to understand the
requirement and be able to make modifications to their protocol
and/or products to meet the requirement.
> > - I am not an operator, but I think if I were one, then
> > - if we were already doing TE measurement stuff (most
> > likely) then reading it seems a waste of time
> > - if we were not yet doing any of it... then I wonder if
> > this would be helpful at all.
>
> Operators have to get by with what is available, what else
> can we do? This does not mean operators are already doing
> 'TE measurement stuff' in an adequate way, and that nothing
> further is required. E.g., see comments above on derivation
> of a traffic matrix.
>
I would hope operators can also understand and support the
requirements. If not, then I really doubt that they are valid
requirements. And they must understand them, so that together
the operators can say to protocol developing WGs and to vendors
that these are the set of requirements to be met.
> > - W.r.t. the TEM WG work item, it says:
> > The tewg interacts with the common control and measurement plane
> > working group to abstract and define those parameters, measurements,
> > and controls that traffic engineering needs in order to engineer
> > the network.
> > So I would expect a CRISP set of "requirements for additional measurements,
> > configurable/negotiable parameters/controls" ... but not the extensive
> > exploration and text that I now see. Why do people (or the WG) think
> > that this document meets the WG deliverable for TEM ??
>
> The requirements are there, and are critically needed. They
> need to be brought out in a clear and crisp way, as suggested above.
>
Yea.. you keep repeating that.
> > - W.r.t. review:
> > - 4 people from ATT support it. Waisum is one of them and is main editor.
> > Others (Nick Duffield, Bob Cole) have some of their material listed in
> > the doc.
>
> So AT&T is stuffing the ballot box?
That was not what I am saying. My point was more: it is clear that those
people who have contributed support the document. Would be strange if they
did not, would it not? But I can see now that maybe I should have made
my statement a bit different.
> I'm mentioned in acknowledgements, others in references, are we
> disqualified to comment? A certain set of people (*individuals*)
> who are the ones most interested/knowledgeable are the ones also
> most likely to comment.
>
> > - 3 EDU users commented, 2 said they found it a good doc
> > 3rd one asked a few questions
> > - Raymond Zhang is positive. He is from info.net ???
> > is that an operator?
>
> yes.
I thought so but was not sure. Maybe Raymond can answer himself, no?
Raymond, can you fill me in what sort of network you operate?
Feel free to do so privately if you prefer that.
>
> > - Richard Tibbs gave a thumbs up, he is from oakcitusolutions.com.
> > What role/function does he play/have? Operator, code/tool-developer?
>
> co-author.
>
Ah... ok, so obvious that he would support it. Should have been in
my first list of people above.
You did not answer (but better if Richard does himself) if he is
and operator or what role plays in his daily life. The reason I ask
this is because I want to udnerstand their background and the eyes
and context that they probably have used to review.
> > - One hotmail user (Spyrokontigiogis) gave a thumbs up. Not that
> > he/she added any comment. Do we know him/her?
> > What role/function does he/she play/have? Operator, code/tool-developer?
>
> Level-3 employee?
>
Can you be more specific? Is he/she in the accounting dept? I guess not.
But again, I want to understand the context within which the review was done.
> > - Dimitri Papadimitrou (Alcatel) asked a question/suggested some text.
> > I did not see if he likes the doc or not
>
> did not object.
>
> > - Blain Christian (uunet, so maybe a real operator?)
> > withdrew as co-editor
> > That does not sound good (in my view)
> > - So where are the real operators that support this?
>
> So *individuals* from AT&T, Infonet, France Telecom, and
> Level-3 (?) supported this, IETF does not have
> companies/operators positions. Also, are you saying that the
> above operators are not 'real' operators versus other
> operators who you think are real operators?
>
Sorry for the bad wording. When I use company names, I try to
see how many differnt viewpoints from people active in different
types of networks have looked at it. From their operational experience.
I do not consider them as speaking for their company, but indeed as
individuals. I'd like to see comments/review/support from all
types of operators, that is carriers, telecom operators, cors ISPs,
Access ISPs, small ISPs, Enterpise Networks... etc.
> > My thought is that I can do two things:
> >
> > - send back to WG and say that this is "not good enough" and
> > that I do not
> > feel comfortable to present it to IESG for approval.
> > It does NOT meet (in my eyes) the WG deliverable for TEM.
> > - send it to IESG with my recommendation to NOT approve for
> > same reasons.
> >
> > So let me try the first option first and ask the WG what they
> > have to say to my review.
>
> I find it rather disappointing that you waited this long to
> come down on this I-D so hard. If you had this level of
> concern, I think you should have tipped your hand long ago,
> but perhaps this is first time you have read the draft.
That is a valid complaint. We are aware that a lot of the AD/IESG level
review takes place too late in the process. Not easy to fix, we (ADs)
all have trouble to keep up with what we have on IESG agenda, let alone
to dive into many WG docuemnts early. We try to improve... but in this
case I missed it.
Now I do understand that the WG chairs have been pushing back on this
in the beginning quite a bit too. I believe that the doc has improved
quite a bit since the earlier versions... but when I review it in
the context of the WG deliverable, then I am clearly still very
dissappointed. If you say that all the "requirements" are hidden in
that doc, then pls try to make them clear. It seems that needs major
rework.
> We have been admonished many times to read the draft and comment
> on this list, and many folks have abided by your wish.
>
I guess it depends on what you count as "many". When I was a kid,
we joked about people counting: 1,2,3,many. Mmm.. maybe that statement
is not appropriate.
> If we discard this I-D, the TEWG charter item also will go
> unfulfilled. That's bad. I think the I-D serves an important
> need, and can address your comments, as above.
>
But I'd rather have us confess that we were unable to deliver on a
WG item than us delivering something that is not in good shape
(or bad quality, which I think is the case).
Hope this explains my posting a bit better.
Bert