Guidelines for the TPC and Reviews

Presentation slides from the INFOCOM 2015 TPC Informational Meeting, Toronto, Ontario, Canada, held April 30, 2014

All Area Chairs, regular and advisory TPC members are invited to attend the TPC meeting on November 8, 2014, to be held at Philadelphia, PA, on the Temple University campus.

Review QualityArea ChairsRegular TPC MembersAdvisory TPC Members

Guidelines on the quality of INFOCOM reviews

Reviews with high quality is of utmost importance to IEEE INFOCOM, and the most important contributions that a member of the TPC can make to INFOCOM. Please avoid submitting very short reviews, and do not cut and paste words and sentences across different categories. If any of the papers are reviewed with the help of graduate students, we expect a TPC member to provide sufficient guidance and oversight, and personally submit the reviews to EDAS.

In the TPC meeting, a review with a score of 5 with a short unsubstantiated review will be discounted, and the other dissenting reviews may override the higher score, if they are more substantive. Try not to be overly negative, as any research paper has its advantages and drawbacks.

Try to write a review with the quality that you would expect for your own papers. The Area TPC Chairs and the TPC co-chairs will read the submitted reviews, and monitor their quality.

Responsibilities of the Area TPC Chairs

As an Area TPC Chair in INFOCOM 2015, you are a member of a distinguished group of experts who are essential to maintaining the high quality of INFOCOM 2015. You are responsible for overseeing the review process of the papers assigned to you.

Step 1. Assign one review to each paper.

The first step is for you to check the papers assigned to you, and to assign one reviewer to each of these papers. In INFOCOM 2015, we have developed and deployed a new automated assignment system that maximizes the total similarity scores between TPC members and submitted papers. To compute the matrix of similarity scores, we compare the published papers from TPC members to all submitted papers. This system has also been used to assign papers to all Area TPC Chairs, with a maximum workload of 39 papers, and a minimum workload of 32 papers.

Here is how you can access your papers:

1. Login onto EDAS (http://edas.info/)

2. Click on "My TPCs" (first row near the top)

3. Click on "INFOCOM 2015".

4. Click on the "Papers" tab (first row near the top)

5. Click on the "List & export" tab (second row from the top)

6. Click on the "List Papers" button near the bottom.

Please first take a moment to check all the papers assigned to you, and let us know if:

1. You know of a Conflict of Interest you have with a paper (in the unlikely event that we have missed it). (Since the submissions are double-blind, this would only apply if you already know of the work.)

2. A paper is clearly out of scope for INFOCOM. You recommend that the paper be rejected without review.

3. A paper violates the double-blind submission policy. You recommend that the paper be rejected without review.

Next, please click on each paper in the list, then click the little purple circle with a plus sign in the "Reviews" section to add a reviewer. You will then be presented with a list of TPC members with matching interests. You can choose from this list but you do not have to; you can instead enter the name, email, or EDAS ID of any reviewer that you believe is the most suitable.

Since the automated assignment system relies on similarity scores that may not be perfect, and due to the inherent mismatch between the topic distribution of submitted papers and the expertise distribution of the Area Chairs, some papers assigned to you may not fall into your area of expertise. Please note that you do not have to assign a review for all papers -- you can skip a paper if you do not feel comfortable assigning a reviewer for it.

Step 2. Oversee the discussion phase and intervene if the TPC Lead is not effective.

During discussion phase, add additional reviewers when necessary, chosen from advisory TPC members. You can enter your own review and/or ask reviewers to modify their reviews/scores to be consistent with the discussions among TPC members during the online discussion phase, the in-person TPC meeting, and in the final recommendation.

Step 3. Provide your recommendations to the TPC co-chairs. (Due date: October 26, 2014)

There are three alternatives: accept, reject, and discuss at the TPC meeting. Make sure the TPC-Lead summary is consistent with TPC reviews and online discussions.

Step 4. Participate in the TPC meeting and lead group discussions.

Responsibilities of the Regular TPC Members

After the Area TPC Chairs assigned a review to each of the submitted papers, all the remaining reviews will be automatically assigned by the new review assignment system, based on the result of globally optimizing the sum of similarity scores across all review assignments, while the similarity scores are computed by comparing, pair-wise, the full text of the submitted paper and TPC authored papers.

A regular TPC member at INFOCOM 2015 is responsible for providing high quality reviews to the assigned papers (around 12-15) by October 3, 2014 — with around 44 days in the review period. Each regular TPC member will also serve as the TPC-lead for about 5 papers, and should actively participate in both the online discussions and discussions at the TPC meeting. Each regular TPC member will be the TPC Lead for about 5 papers, and will be a regular reviewer for about 12-15 papers (including the ones for which he/she is the TPC lead reviewer).

For papers for which you are the TPC Lead, in your EDAS screen (after you login, select "My TPCs" and INFOCOM 2015 from the list of conferences), you will see both a link to tpc as well as review under the "Type" column. For those papers for which you are a regular reviewer, you will only see the review link under the "Type" column.

First, you need to provide a review for each of the papers in your list by using the "review" link and filling out the review form by October 3, 2014. You may delegate up to 5 regular reviews to external reviewers (who are not students) with sufficient expertise. Delegations need to be completed by August 31, 2014. You must follow up with their delegates to finish their reviews, and oversee the review quality.

Starting October 3, 2014, there will be an online discussion phase, during which you can discuss with other reviewers/TPC leads the outcome of each of the papers that has been assigned to you for review. The delegates are also responsible for actively participating in the online discussion phase. The messages exchanged during the online discussion phase will not be made available to the authors.

Once the discussion phase is over, for those papers that you serve as the TPC lead, you will need to provide a summary of the discussions in the form of a meta review Please click on the tpc link under the "Type" column for your TPC lead papers, fill out the TPC summary form, and choose one of four categories for each paper: accept, discuss, needs additional review, reject. The TPC lead recommendations (meta reviews) should be completed before October 25, 2014. They will not be made available to the authors.

Responsibilities of the Advisory TPC Members

An advisory TPC member at INFOCOM 2015 will be assigned papers by the TPC co-chairs and the Area TPC Chairs. Advisory TPC members will play a critical role in the review process, by providing high quality reviews in a very short time span. These reviews are typically requested due to inconsistency in existing reviews. Advisory TPC members should also actively participate in the online discussions and discussions at the TPC meeting.