Ideosphere Forum
From owner-fx-discuss@ideosphere.com  Wed Oct 11 19:36:07 2017
Return-Path: <owner-fx-discuss@ideosphere.com>
Received: from ideosphere.com (localhost [127.0.0.1])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id v9BNZtco004527
	for <fx-discuss-outgoing@ideosphere.com>; Wed, 11 Oct 2017 19:35:55 -0400
Received: (from majordomo@localhost)
	by ideosphere.com (8.13.8/8.13.8/Submit) id v9BNZtk1004526
	for fx-discuss-outgoing; Wed, 11 Oct 2017 19:35:55 -0400
Received: from smtpq1.tb.ukmail.iss.as9143.net (smtpq1.tb.ukmail.iss.as9143.net [212.54.57.96])
	by ideosphere.com (8.13.8/8.13.8) with ESMTP id v9BNZp0T004515
	for <fx-discuss@ideosphere.com>; Wed, 11 Oct 2017 19:35:54 -0400
Received: from [212.54.57.81] (helo=smtp2.tb.ukmail.iss.as9143.net)
	by smtpq1.tb.ukmail.iss.as9143.net with esmtp (Exim 4.86_2)
	(envelope-from <chrisran.bma@virgin.net>)
	id 1e2QXW-0005i1-Q9
	for fx-discuss@ideosphere.com; Thu, 12 Oct 2017 01:35:50 +0200
Received: from oxbe10.tb.ukmail.iss.as9143.net ([172.25.160.141])
	by smtp2.tb.ukmail.iss.as9143.net with bizsmtp
	id LPbq1w00433LN9o01Pbq0L; Thu, 12 Oct 2017 01:35:50 +0200
X-SourceIP: 172.25.160.141
X-Authenticated-User: chrisran.bma@virgin.net
Date: Thu, 12 Oct 2017 00:35:50 +0100 (BST)
From: "chrisran.bma e-mail" <chrisran.bma@virgin.net>
To: fx-discuss <fx-discuss@ideosphere.com>
Message-ID: <208534181.5060817.1507764950761.JavaMail.open-xchange@oxbe10.tb.ukmail.iss.as9143.net>
In-Reply-To: <922163720.464822.1482229147840.JavaMail.open-xchange@oxbe4.tb.ukmail.iss.as9143.net>
References: <1040866937.153983.1481290222057.JavaMail.open-xchange@oxbe2.tb.ukmail.iss.as9143.net> <CAD=nd8fYkmi+_Y6Uc3zsxynyF49PA7+x-zg-xSmOMFOmj0Obvg@mail.gmail.com> <CA+EdDHhY48A_EDBL+aZvbC00qA4075k6PoUZwg=HXCL_0vS=LQ@mail.gmail.com> <922163720.464822.1482229147840.JavaMail.open-xchange@oxbe4.tb.ukmail.iss.as9143.net>
Subject: Re: fx-discuss: FX Claim Tran - Machine Translation by 2015
MIME-Version: 1.0
Content-Type: multipart/alternative; 
	boundary="----=_Part_5060816_460236630.1507764950747"
X-Priority: 3
Importance: Medium
X-Mailer: Open-Xchange Mailer v7.6.2-Rev60
X-Originating-IP: 78.147.44.85
X-Originating-Client: com.openexchange.ox.gui.dhtml
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=virgin.net;
	s=meg.feb2017; t=1507764950;
	bh=6yoWQv1TRaBS+y5DXZ3j2qfBaXD23Wv79oTPOn4xld8=;
	h=Date:From:Reply-To:To:In-Reply-To:References:Subject;
	b=Q8MwWDIEPva6Bn4THDgo77L/fkX+CZqk7mxvTi4ttmnc/uWFsqpsODxQxxVknkBdC
	 kc5jP3RTUlb0HnaD5ecLXKErj1vO2sf42tc9BxAZCutDTGNi7SQsKkBWaOcLJWUkl0
	 xodXAjjpJmwv2sgPi5yyQQeX2t/mYi0j7n7m64h3+b4tzhtcMf9r9iAA7jCD+NAN4r
	 WfP5ViiBDFzfFVzyRTawovzfZQFXUTikdMhu8vhxqep+eKl4INjWne2x+O8zCJBn+g
	 xS2h7kSm6XsxdJHBoCoavmqKEtq8cgUf36lLoa0CbygktdqSg3z6WKjmUFhoclk1pl
	 KrJ775FbCTrBQ==
Sender: owner-fx-discuss@ideosphere.com
Precedence: bulk
Reply-To: fx-discuss@ideosphere.com

 
 Possibly relevant to claim Tran - Machine translation by 2015:

Findings of the 2017 Conference on Machine Translation (WMT17)

http://www.statmt.org/wmt17/pdf/WMT17.pdf

(September 7-8, 2017)

and

Findings of the 2016 Conference on Machine Translation (WMT16)

http://www.aclweb.org/anthology/W16-2301

(11-12 August 2016)

Perhaps the following looks like what we want:

[quote]

5.5.2 Human evaluation results

Table 35 includes DA results for English-German and Table 36 shows results for
German-English APE systems. Clusters are identified by grouping systems together
according to which systems significantly outperform all others in lower ranking
clusters, according to Wilcoxon rank-sum test.

# Ave % Ave z System

___________________________

− 84.8 0.520 HUMAN POST EDIT

___________________________

1 78.2 0.261 AMU

   77.9 0.261 FBK

   76.8 0.221 DCU

___________________________

4 73.8 0.115 JXNU

___________________________

5 71.9 0.038 USAAR

   71.1 0.014 CUNI

   70.2 −0.020 LIG

___________________________

− 68.6 −0.083 NO POST EDIT

Table 35: EN-DE DA Human evaluation results showing average raw DA scores (Ave
%) and average standardized scores (Ave z), lines between systems indicate
clusters according to Wilcoxon rank-sum test at p-level p ≤ 0.05.

 [/quote]

Seems to indicate that human translation is better than machine translation, but
of course that doesn't guarantee that there isn't a better translation program
somewhere from pre 31 Dec 2015 that simply didn't attend the conference.

Still if human level translation existed in 2015, you would not expect to read
things like

[quote]

This steady improvement has been mainly driven by the massive migration to the
neural approach, which in 2016 allowed the winning system to achieve impressive
results

[/quote]

I don't believe there is a program that can justifiably claim "equal or better
average quality, as professional human translations" but proving a negative is
difficult. I suggest if there was such a program it would be big news, not
difficult to find, and conference findings would be markedly different to those
linked above.

Not sure how much more a judge might want before deciding how to judge the
claim. Are there any more authoritative events or other event before claim
deadline of 31 Dec 2017? (Note program has to exist by 31 Dec 2015 and
translations have to 'be of comparable cost and turnaround time'.)

Regards

Chris

(crandles 7886)

Disclosure I hold -3603 in this claim

Back to regular message display

All trademarks, copyrights, and messages on this page are owned by their respective owners.
Forum: Copyright (c) 2000-2001 Javien Inc All rights reserved. Distributed under the GPL