Workshop Karlsruhe

From Swsc-WIKI
Jump to: navigation, search

Sti-int.png Sap.jpg

This workshop has been sponsored by Semantic Technology Institute (STI) International and SAP AG

7th Semantic Web Services Challenge Workshop

Part of the 7th International Semantic Web Conference ISWC 2008

October 26, 2008

Kongresszentrum (Congress Center) in Karlsruhe, Germany

Here you will find the previous SWS-Challenges: SWS-Challenge

Overview and Introduction

Research of Semantic Web Services has gained a lot of momentum over past few years, demonstrating its viability for dynamic service discovery, selection, composition, negotiation, mediation and invocation over the Web. However, design and development of Semantic Web Services tools and applications is still a new practice not only with the research community but also the industry. Software developers need to deal with new kinds of artifacts (e.g. semantic descriptions, ontologies, complex rules, formulas and facts), new kinds of operations (e.g. mapping, merging, transformation, querying and reasoning) and probably new kinds of user interactions and visualizations.

Best practices, methodologies and tools to help in dealing with these artifacts and operations in Semantic Web Services development are thus in high demand. Moreover, there are few scientific method of comparing and evaluating the strengths and weaknesses of functionalities claimed. Other than the SWS Challenge, there is no standard testbed to assess the robustness and applicability of proposed technologies to real world problem scenarios. Progress in scientific development and in industrial adoption in Service Oriented Architectures (SOAs) is thereby critically hindered.

The SWS Challenge initiative is specifically designed to develop a standard methodology and testbed for the evaluation of Semantic Web Services technologies. In addition to this this initiative, there is the associated W3C SWS Testbed Incubator which has issued a W3C Incubator Group Report about the evaluation methodology developed within the SWS Challenge.

This workshop, under the umbrella of the ISWC 2008 conference will provide a common platform for the researchers and developers of the Semantic Web Services to present and discuss the latest progress in the area, to identify the gap between the Semantic Web Services research and required methodologies, tools as well as the applications.

First, it will focus on finding out different ways, methodologies, tools and technologies for evaluation of Semantic Web Services tools and applications, continuing the work of the SWS Challenge, and the Incubator. Participants should be familiar with the previous work of these initiatives, including the Incubator Group Report above. Papers on this subject are highly welcome, and need not be accompanied by a Challenge solution to be evaluated.

Thus, participants are invited to discuss the Challenge's current evaluation methodology. Insights gained during the workshops will provide additional input to the W3C SWS Testbed Incubator Group which aims at standardizing an evaluation methodology for semantic web services technology.

Second, the workshop will continue the SWS Challenge which is an ongoing and continuous experiment in evaluating the functionality of SWS technologies. The curent set of evaluations are posted on this wiki and proposed solutions will follow the current evaluation methodology also available here. Participants are invited to propose new scenarios to be incorporated into the official SWS-Challenge testbed.

The activity most closely related to the SWS Challenge is the S3 Contest on Semantic Service Selection. However, the SWS Challenge and the S3 contest are working in different parts of the reseach space. The organizers agree that they are related and both should be explored. In fact, the cross-referencing between both events on their web sites and inter-linkage of other activities (e.g., common use of test services) has been agreed between organizers.

Topics of Interest

SWS Evaluation Methodolgy

  • Methods to describe evaluation aspects and parameters
  • Identification of resource consumption characteristics
  • Models for describing and measuring in evaluation
  • Techniques and formulas for evaluation
  • Infrastructures and tools for evaluation
  • Performance and scalability evaluations and benchmarks
  • Integration of evaluation tools into frameworks
  • Description of application of SWS technology in industry or on
  • real world use cases and evaluation metrics

Semantic Web Services Challenge

  • Descriptions of implemented solutions to any of the SWS-Challenge problem scenarios
  • Proposals of new scenarios for evaluation
  • Evaluation of surprise scenario variations

Target Audience

The Evaluation Methodology track of the workshop will address the researchers and practitioners working in the evaluation and benchmarking of Semantic Web Services technologies. It will also target the potential users to learn the feasibility of Semantic Web Services to be used in market and industry.

The actual Challenge Evaluation track will address the researchers and practitioners developing evaluation scenarios in context of Semantic Web Services challenge as well as building solutions for the given problem sets. The previous workshops have attracted not only people from the semantic web community in general, but also from the formal methods and software engineering community and from industrial research. The Challenge has thus succeeded to bridge communities working on different aspects in the field.

Preliminary Workshop Program

The precise program is yet to be announced. This preliminary program is subject to change.

Sunday, 26.10.08

  • 09:00: Introduction of people and process
  • 09:30 - 10:00 Dick A.C. Quartel, Stanislav Pokraev, Teduh Dirgahayu, Rodrigo Mantovaneli Pessoa, Marten van Sinderen: Model-driven Service Integration using the COSMO framework
  • 10:00 - 10:30 Jens Lemcke, Matthias Kaiser, Christian Kubczak, Tiziana Margaria, Björn Knuth: Advances in Solving the Mediator Scenario with jABC and jABC/GEM
  • 10:30 Break
  • 11:00 - 12:30 Code review of COSMO, jABC/GEM, jABC, LTL solutions (updating the evaluation matrix as necessary)
  • 12:30 Lunch
  • 14:00 - 14:30 Presentation/Discussion of New Scenario Proposal
  • 14:30 - 15:00 Discussion / Future Plans / Closing of public part
  • 15:00 - 18:00 Evaluation of solutions to Surprise Problem:

attendance restricted to solution attemptees and organizers.

Monday, 27.10.08

We recommend attendance at the Service Matchmaking and Resource Retrieval in the Semantic Web (S3 Contest) Workshop.

Of particular interest with respect to the SWS-Challenare is the talk by Ulrich Kuester and Birgitta Koenig-Ries at 10:05 and the invited talk by Holger Lausen at 12:00.

There will be a plenary session of the SWS Challenge, at the conference venue (exact location will be announced during the 1st day of workshop), at 14:00 to discuss the location of future workshops and various process and management issues. All interested parties are invited.

At 16:00, we recommend participation in the S3 Contest Workshop's general discussion on evaluation issues.

Organizing Committee


PC Chair

Scientific Organizing Committee

  • Omair Shafiq, Semantic Technology Institute Innsbruck (AT)
  • Federico Facca, Semantic Technology Institute (STI) Innsbruck, Austria

Scientific Advisor

STI2 Liaison

Technical Support

Program Committee

Submission and Proceedings

Submissions are to be formatted according to the guidelines of the ISWC2008 conference (Springer LNCS style). The page limit for full papers is 10 pages, for SWS-Challenge solution papers two additional pages are accepted.

Papers describing SWS-Challenge solutions should describe the technology employed for the solution and focus on how that technology was actually used to solve the challenge problems. The reader should be able to understand a particular solution in theory and practice by reading a solution paper. Additional submission of the code and documentation of the solution is strongly encouraged.

Note that ISWC requires workshop attendees to also register for the main conference. So try to register both for the conference and the workshop before the ISWC2008 early registration deadline.

Since the ISWC does not include workshop proceedings, we intend to publish the workshop proceedings online at CEUR-WS. If the number of high-quality submissions is high enough, we will pursue publishing the proceedings as a journal special issue or as printed proceedings.

Guidelines for Paper Submission

Papers are to be submitted online via the SWSC-ISWC2008 EasyChair submission system.

In case of any further questions about submission process, contact Omair Shafiq (

Guidelines for Code Submission (only for SWS Challenge solution papers)

Interested participants should directly contact Srdjan Komazec ( to get an account for the SWS-Challenge FTP server (

Important Dates

  • Workshop CFP will be out: 2 May 2008
  • Submissions due: 25 July 2008
  • Acceptances: 5 September 2008
  • Camera ready papers: 26 September 2008
  • Camera ready proceedings: 03 October 2008
  • Workshop date: 26 October 2008 (1st workshop/tutorial day)

Please do not hesitate to contact Charles Petrie for any questions you have!


Registration for ISWC and its workshops is now open. See more information and a URL to the registration system at

ISWC requires workshop attendees to also register for the main conference. So try to register both for the conference and the workshop before the ISWC2008 early registration deadline.

Hotel and Local Information

Local information including hotel information and special conference rates are available through the ISWC 2008 website.

Additionally, a participant of the workshop kindly recommended this site that has a students/teachers rebate scheme.

Workshop Dinner

A reservation for 10 people have been made at a traditional local restaurant. This event was sponsored by STI Innsbruck. Details are as follows:

 Date: Sunday, 26th October 2008
 Time: 19:30 hrs 
 Place: Der Vogelbräu Karlsruhe
      Kapellenstraße 50
      Tel 07 21/ 37 75 71
      Fax 07 21/ 37 09 02 
 Reservation reference at the restaurant is
 Name: Shafiq
 Event: SWS Challenge

Surprise Problem Agenda

Date/Time Phase Description
(1) 20/10/2008 20:00 CET "Code freeze" credentials distribution Committed participants will receive instructions and credentials for the Phase 2.
(2) 22/10/2008 20:00 CET "Code freeze" submission The deadline for existing solution submissions.
(3) 23/10/2008 08:00 CET Surprise problem announcement Committed participants will gain access to the surprise problem description.
(4) 25/10/2008 20:00 CET Solution submission The deadline for surprise problem solution submissions.
(5) 26/10/2008 08:00 CET Solution verification The surprise problem solutions verification report.

Please note that a solution submission (Phase 4) should be accompanied with a document clearly stating all changes that were introduced to the frozen version in order to respond to the surprise problem requirements.

Certification Results

The table below shows the results of the certification of the solutions presented at this workshop. Please note that the table reflects only the solutions or solution aspects certified at this workshop. For a complete overview of the certifications provided within the SWS-Challenge, please refer to the tables on the main page of this wiki. For detailed information about the individual solutions and to access the available technical content, please visit the solution overview and documentation page or use the links in the table header below.

Evaluation Results for the Mediation Scenarios
Problem Level Telematica Instituut & University of Twente
(Solution Details)
University of Dortmund & University of Potsdam (Pure jABC)
(Solution Details)
University of Dortmund & University of Potsdam (LTL Synthesis)
(Solution Details)
University of Dortmund & University of Potsdam & SAP Research
(Solution Details)
FSU Jena
(Solution Details)
0: Static mediation (Purchase Order Mediation) 1 2 1 1
1a: Changes data mediation (POM v2) 1 1
1b: Changes process mediation (POM v2) 1 1
1c: Payment Problem 1
1d: Mediation Surprise3 √+ √+ √+ √+

1 Solutions presented at previous workshops
2 Solution described in Innsbruck Workshop Paper
3 A checkmark indicates successful participation in the surprise scenario. A + character additionally indicates that the solution was flexible enough to allow adaptation to required small changes in the scenario at the workshop. Details on the Surprise scenario are not revealed in order to allow new participants solving it as a surprise scenario later (see Surprise Problem Methodology).