Glyphius… After 181 Test Rounds How Does it REALLY Stack Up?
At the beginning of each year I like to look back on the testing I’ve done over the previous 12 months and see what information, trends and surprising results I can discover within the data.
Now… I usually reserve sharing the results of my annual review with my $2,500 per month Platinum Coaching clients and the small group of world-class millionaire marketer buddies who subscribe to my closed circulation, printed “Conversion Doctor Letter”. (Both programs are by invitation only, and currently full. So please don’t ask…)
But, one of the data groups I stumbled across was surprising…
While I’ve done FAR more than just 181 different headline test rounds over the past 12 months, I did 181 rounds where I was specifically testing a higher Glyphius score as the primary differentiating factor in the headline.
For those of you who are not familiar with Glyphius, it is a piece of software developed by my friend (and cruise buddy) James Brausch that mathematically validates your copy against copy that has been tested and proven profitable.
It basically compares your copy against the thousands of ads that are stored in its database to determine if your copy is similar to known profitable copy.
So you enter your headline and Glyphius assigns a numeric score. Then you tweak the copy and re-score it, to try and get a higher score. (Copywriter Brian Keith Voiles says it’s like playing a video game.)
In fact when I am running these “higher Glyphius score” test rounds I often name my headline variables by their Glyphius score.
Here is an example:
(So if I’ve run test rounds for you and you saw me using those “strange” variable names like G420 or G691, now you know why!)
When James came out with Glyphius there was quite a bit of controversy about the effectiveness of the software, but one by one many top copywriters (myself included) began endorsing it after we put it to the test and discovered that the darn thing actually worked pretty good!
Well… My long term test results were actually BETTER than my initial few rounds.
Here is how the 181 test rounds turned out:
Higher Glyphius score BEAT the control: 127 times
Higher Glyphius score TIED* with the control: 21 times
Higher Glyphius score LOST to the control: 33 times
* (In reality if you let a test round run long enough a “winner” will eventually emerge, but the very small improvement is usually not worth allowing a test round to run for an excessive length of time. You’ll wind up leaving far more money on the table by delaying your next round.)
So… Glyphius actually “predicted” the winner in over 70.1% of the tests.
But what really surprised me was the low number of “ties”. From my experience in most multivariate test rounds around 1/3 of the variables you test will not produce a measurable impact one way or another. (Unless you run the test “forever” like I mentioned.)
So based on that I had expected to see around 60 “ties” out of 181 variations tested.
I’m still scratching my head a bit to figure out why there were not more rounds that came up about even.
The only explanation I can think of is that different words (even synonyms) carry a vastly different emotional punch and produce different responses. So when trying for a higher Glyphius score, perhaps you are modifying and mixing enough of these “high response” words into the headline that it is significantly changing the emotional “charge” of the copy enough to either boost or hurt response, but not leave much room in between. Who knows?
Bottom line is… I’ll be using Glyphius even more when creating variables to test in 2008.
If you want to learn more about Glyphius you can visit http://www.glyphius2007.com (And NO that’s not an affiliate link.)
Or you can also get a copy from my friend Ryan Healy, and also get a copy of Muvar (James’s multivariate testing software) as part of the package.
I have not played with Muvar enough to give it my full endorsement yet, but my initial experiences with it are promising. I particularly like it for doing price testing because it’s one of the only testing tools that automatically calculates visitor value, rather than just conversion rates.
You can grab the combo pack from Ryan at: http://ryanhealy.typepad.com/copywriting/2007/11/swiping-split-t.html
Also, on a related note…
This Monday (Feb. 4th) at 8:00 pm Eastern I’m conducting a special FREE Webinar where I am going to share some of my other “Headline Testing Secrets” discovered over the past year.
Want to know…
- What LENGTH headlines are converting best?
- What COLOR worked best in 2007? (And I’ll bet it will do great this year too!)
- What FONT TYPES pull better?
- What WORDS almost always boost sales?
- How to test up to 24 headlines (and combinations) at the same time? (WITHOUT having to run the test forever just to see the results!)
- And a ton more…
Then lock in your limited, FREE spot (there are only 500 available) at:
http://www.HeadlineTestingSecrets.com/blog-post/
UPDATE!
It looks like this post on Saturday completely sold out all of Ryan Healy’s inventory of Glyphius! And from what I can tell James is not selling Glyphius directly any more, he is selling it through a network of resellers.
However if you visit www.JamesBrausch.org you’ll see a link on the right side to one of his resellers who still has inventory.
Also…
Just in case you haven’t already registered for tonight’s FREE “Headline Testing Secrets Revealed” webinar we are now down to only a couple hundred spots left.
So if you want to attend, you had better lock in your spot right now!
Visit: http://www.HeadlineTestingSecrets.com/blog-post/
Popularity: 37% [?]
Related Posts:
February 2nd, 2008 at 9:49 pm
[…] post by The Conversion Doctor Blog and syndicated by Internet Marketing […]
February 2nd, 2008 at 10:07 pm
[…] unknown wrote an interesting post today onHere’s a quick excerptAt the beginning of each year I like to look back on the testing I’ve done over the previous 12 months and see what information, trends and surprising results I can discover within the data. Now… I usually reserve sharing the results of my annual review with my $2,500 per month Platinum Coaching clients and the small group of world-class millionaire marketer buddies who subscribe to my closed circulation, printed “Conversion Doctor Letter”. (Both programs are by invitation only, and currently f […]
February 2nd, 2008 at 10:22 pm
[…] tons more of my most recent headline related test results. You can read the blog post here: Glyphius? After 181 Test Rounds How Does it REALLY Stack Up? | The Conversion Doctor Blog Happy Testing! Eric Graham, "The Conversion Doctor" PS - And don’t worry… The post […]
February 2nd, 2008 at 10:59 pm
[…] may want to subscribe to my RSS feed. Thanks for visiting!Eric Graham just posted at his blog about testing headlines with Glyphius…. And it is exciting news. He tested over a 12 month period. He said the showed that […]
February 2nd, 2008 at 11:00 pm
I’ve had similar results as yours with Glyphius. Except I only did one test. About a year ago a top copywriter posted the results of his split testing. I checked the Glyphius scores against his split testing results and Glyphius was in agreement with his results 85% of the time. Which is the same number James had claimed on his blog. I’ve never doubted the results since that day and “Glyphius checks” are a normal part of my work.
February 2nd, 2008 at 11:09 pm
[…] may want to subscribe to my RSS feed. Thanks for visiting!Eric Graham just posted at his blog about testing headlines with Glyphius…. And it is exciting news. He tested over a 12 month period. He said the showed that 70.1% of […]
February 2nd, 2008 at 11:48 pm
Interesting post - I’ve been wondering about the effectiveness of Glyphius for a while. Unfortunately…neither of your purchasing links are still active.
February 3rd, 2008 at 12:09 am
Hey Eric,
I’m curious — How many actions do you count before deciding that it’s a “tie” that’s not worth letting run “forever?”
ie. In Traffic Secrets, John Reese talked about letting one tests run until one hits 30 actions first. Is this the same criteria you use?
February 3rd, 2008 at 12:54 am
[…] http://www.conversiondoctor.com/conversion-blog/2008/glyphius-after-181-test-rounds-how-does-it-real… […]
February 3rd, 2008 at 3:46 am
If I click the Buy link at http://www.glyphius2007.com/ I get “The product you requested is no longer available”…
February 3rd, 2008 at 9:46 am
Glyphius Predicts Winners 70.1% of the Time?…
I woke up this morning to a pleasant surprise. My email inbox was filled with PayPal receipts.
I sat there scratching my head for a moment… “Did I run a promotion I forgot about?” Well, the answer was “yes,” kind of.
I did…
February 3rd, 2008 at 12:17 pm
[…] http://www.conversiondoctor.com/conversion-blog/2008/glyphius-after-181-test-rounds-how-does-it-real… […]
February 3rd, 2008 at 12:41 pm
“Glyphius 2008″ has no active distributors selling the software at this time, though it has been produced. In the meantime, You can avoid using these words in your sales copy:
ravenous
monoxide
pushdown
venomous
genocide
downside
Clearly “up-beat” copy is better for sales.
February 3rd, 2008 at 1:24 pm
[…] Graham just posted about his results regarding Glyphius and how his testing confirmed the ability of Glyphius to pick the best copy to use for sales language. Some have said this is […]
February 3rd, 2008 at 5:11 pm
Hi Eric
The key problem I found with Glyphius was the lack of creative help you get. It would be good to have some sort of material that helped you generate new headlines. I spent hours going around in circles with nothing coming up.
You know the old saying, if you have crap in one hand and dreams in the other what do you have in your hands?
CRAP.
It doesn’t matter how good the software is, without creative to feed into it, you still have nothing.
But still, it is good to hear that I the software I bought works!
Regards
Justin
www.lazy-internet-marketing.com
February 3rd, 2008 at 6:19 pm
[…] Glyphius… After 181 Test Rounds How Does it REALLY Stack Up?… […]
February 4th, 2008 at 5:40 am
[…] http://www.conversiondoctor.com/conversion-blog/2008/glyphius-after-181-test-rounds-how-does-it-real… […]
February 4th, 2008 at 8:34 am
Discover Headline Testing Secrets!…
Eric Graham - The Conversion Doctor Here is a valuable tip from me. Check this out!
The Conversion Doctor Eric Graham reveals results from his headline testings that he has performed during the last 12 month. You know, that increasing the conversion…
February 4th, 2008 at 9:10 am
Derek,
In response to your question:
In all due respect to John, you want to be careful using “30 actions” (or the popular “rule of 40″) when deciding to declare a winner, loser or tie in a test round.
The key to look for is the DIFFERENCE in actions between your control and variation. The difference is what will give you statistical significance.
For example using the 30 action scenario, lets say that your control has 29 actions and your variation has 30 actions. should you just call the new version the “winner”? Is it statistically significant? Nope…
But, lets say that after only 20 actions the control has 19 actions and the test version has only 1 action. Should you keep the test running just to get those 10 or 20 extra actions to hit your magic number of 30 or 40?
Of course not. The new version is obviously a dog and you are losing sales every day that test is running.
Now, when to call something a “tie” is a bit different. In almost all of my testing I am running multivariate rounds, so I typically have anywhere from 3 to 8 different sections being tested at the same time.
So if most of my other variations have hit a wide enough difference to determine the actual “winners”, but I still have one that is “too close to call”, I’ll usually declare it a tie and end the test.
But if you are REALLY looking for me to give you a specific number to use in calling something a “tie”…
For me it depends on what “action” I’m trying to optimize for.
If I am working to improve optins, I like to see at least 150 to 200 total actions. And if I am testing to improve sales, I shoot for 50 to 100 actions.
Hope this helps!
Eric
February 4th, 2008 at 9:19 am
Justin,
You are exactly right…
A tool is only as good as the creativity (or skill) of the user.
One of the factors in the testing process that does not get discussed nearly enough is the impact that the quality of the variables you test has on the final result of your testing.
It’s the old Garbage in Garbage out principle.
If you test junk variables, you’ll get junk results.
That is why when testing (particularly headlines) it’s critical to get a firm foundation on your headline copywriting skills FIRST.
You need to be able to write (or at least recognise) strong headlines before you even begin testing.
Warning: Shameless Self Promotion Ahead…
If you want to pump up your headline writing and testing muscles, I strongly recommend you attend my FREE webinar tonight!
http://www.HeadlineTestingSecrets.com
Happy Testing!
Eric
February 4th, 2008 at 11:47 am
I wondered about the 30-40 rule and looked it up. Its the point at which your split test results are 85% or more accurate. As you said Eric, if the difference is small at that point, it’s time to move on to another test anyway.
With multi-variate testing, you just let the test run and the testing optimizes itself forever. At least that’s how “MuVar” works.
February 4th, 2008 at 11:49 am
[…] Glyphius… After 181 Test Rounds How Does it REALLY Stack Up? By Eric Graham, “The Conversion Doctor” Glyphius At the beginning of each year I like to look back on the testing I’ve done over the previous 12 months and see what information, trends and surprising results I can discover within the data. Now… I usually reserve sharing the … The Conversion Doctor Blog - http://www.conversiondoctor.com/conversion-blog […]
February 4th, 2008 at 2:30 pm
I don’t really like James Brausch… basically because he called me a fraud… and he was wrong.
But, even I have to admit that some of his products, not all, deserve the accolade they’ve been receiving.
Glyphius and Muvar are two which spring to mind.
I use these to good effect and will be using them even more in the future.
Rezbi
February 15th, 2008 at 3:26 pm
[…] http://www.conversiondoctor.com/conversion-blog/2008/glyphius-after-181-test-rounds-how-does-it-real… […]
March 3rd, 2008 at 5:21 pm
[…] Glyphius… After 181 Test Rounds How Does it REALLY Stack Up? […]
March 3rd, 2008 at 8:18 pm
Hi Eric
I did check out your webinar…. can’t get there again as I have already signed up though.
The reason I did not jump in is because of money.
What would be good though is a list of synonyms to be using with Glyphius to help the creative juices flow.
I tend to use a swipe file of old headlines but Glyphius keeps telling me I am using bad words. So having synonyms that work for those bad words would be good. Also a way to restructure the headline instead of using something that doesn’t work.
Regards
Justin
April 4th, 2008 at 1:09 pm
“What would be good though is a list of synonyms to be using with Glyphius to help the creative juices flow.”
I made some software directly for that purpose, currently in Beta testing. If you grab a page of synonyms from Thesaurus.com or similar, Glyphius will “eat” through a page of text in 10 seconds. You simply sort the results and you have a list of Glyphius approved synonyms.