assessment of proposals

Assessment Criteria

The Assessment Criteria are primarily concerned with the relevance, significance and quality of the work represented in the proposal. Separate criteria are used for the assessment of the proposed conference presentation and the workshops, both pre-conference and mini workshops.

The criteria for assessing presentations are the following:

  1. Relevance to evaluation of the focus of the proposed presentation;
  2. Significance for evaluation of the issue or subject focused upon;
  3. Originality of the solution or approach taken to the issue;
  4. Extent to which the proposal reflects a sound conceptual basis and argument in relation to the issue(s) addressed;
  5. Extent to which the conceptual argument and the research and evaluation methods used (qualitative, quantitative, analytical, etc.) are sound given the nature of the task;
  6. Relevance to the overall conference theme and to one or more of the sub-themes; and
  7. Significance for the aspects of evaluation represented by the conference theme and sub-themes.

Where a proposal does not directly address one of the conference themes acceptance and scheduling decisions will be based upon Criteria i) to v) together with the overall purposes of the conference. Most conferences accept some proposals which are situated outside of the conference theme(s).

The criteria for the assessment of workshop proposals are the following:

  1. Relevance to evaluation of the focus of the proposed workshop;
  2. Significance for evaluation of the particular knowledge, techniques and practices focussed upon;
  3. Extent to which the proposed teaching/learning strategies and resources are sound and appropriate for the topic and target group specified;
  4. Extent to which the proposed objectives and prerequisites are clear and appropriate for the target group specified, and the objectives are achievable; and
  5. Overall value for the target group specified.

Assessment Procedure and Rationale

Each of the Assessment Criteria has been selected to reflect an important and essential aspect of good quality and useful conference presentations and workshops. While the utilisation of good communication skills clearly is also of major importance this cannot be adequately assessed through a brief proposal. Where the Program Committee has some misgivings in this regard further enquiries should be made to assist in the acceptance decision.

The assessment procedure which follows is designed to ensure that a fair and defensible process is utilised to determine the proposals to be accepted for inclusion in the conference program. It is based on well established and practical measures and principles for making selection decisions.

  1. Each proposal is to be rated by at least two people working independently (with at least one having expertise in the substantive area of the proposal and one with expertise in the research and evaluation methods used) who are also to provide brief evaluative comments on the proposal. After their initial ratings are completed there should be an opportunity for discussion between them to resolve major differences, but with both their initial and any revised ratings to be reported to the Conference Program Committee.
  2. The ratings are to be made using a five-point scale with respect to each criterion together with a ‘Not applicable’ criterion.)
  3. The ratings are to be used for sorting, but are not to be summed or averaged to get an overall 'score'. This will allow the acceptance decisions to be made by considering the assessments made of each proposals against the criteria in turn.
  4. (NB: A simple additive process would allow the possibility that relatively less important criteria could swamp the other criteria.) In addition,
  5. The sorting of proposals is to be done initially on the most important criterion and then in turn down to the least important criterion as recommended by the Conference Program Committee and confirmed by the Conference Committee.
  6. Final acceptance decisions are to be made by taking into account overall program considerations such as balance across the different conference themes, types of proposal and specific conference purposes, and ensuring that proposals are accepted from a wide range of conference participants. The maximum number of proposals which may be accepted from any one person is three: for example, a pre-conference workshop; a sole paper presentation; and a symposium presentation as part of a team or a poster.

A Reviewer’s Assessment Form is available for presentation proposals and for workshops to allow consistent assessments to be made by the reviewers.  Note that this form will open in a new window and requires a PDF reader to view the form.

Outcome

The decision options are ‘accept outright’, ‘accept subject to revisions’ with relevant feedback being provided, and ‘reject’, again with relevant feedback being provided, given tactfully and taking into account the criteria noted above in terms of balance and levels of expertise.