About Quality improvement in Call of Duty gameplay

The high average quality of products in today’s market is a common sales issue. In other words, the majority of them are sufficient to meet the user’s needs. Manufacturers should seek out or create minor quality differences in these circumstances to distinguish their products from competitors’ offerings and entice customers to switch from an older model to a more modern one.

Making such changes could be expensive and run the risk of not getting the desired response from clients. To minimize this kind of risks producers can use tools of quality such as A/B testing and forecast response of the wide population by testing product on relatively small sample saving budgets under the risk.

The quality of products may be improved with such tools as control and Pareto charts, cause-and-effect and scatter diagrams, histograms, stratification, and check sheets. However, these instruments are applicable in a case of issues existing and detected. However, A/B testing allows making a comparison between products on therefore find out non-detected issues.

This paper contains the description of A/B test applied to regular and elite Xbox controllers in the gameplay call of duty. Therefore, next parts separately introduce the reader to the A/B testing method, the context of call of duty competition world, and Xbox controllers. In the main part of the paper provided a detailed methodology of testing, its results, and reflection on them.

A/B Testing

The methodology of A/B Testing & Optimization

As A/B testing has roots in web-design the basic terminology of A/B Testing

• Element - a discrete unit on the page: a block of text, a form, a button, an image, etc.

• Page - a web page or landing page that is considered the control page for your test.

• Variation - a version of a page that has some changes made to page elements. Also referred to as a variant.

• Test - a hypothesis that one version of an element will change the conversion rate in a significant, hopefully, beneficial way.

• Conversion - when a visitor takes a desired action on the page (Unbounce).

The adaptation for products such as game controllers will include other entity (product) then the page.

Next discussion of A/B test will include the form adopted for goods testing, not a web page testing.

The process of running A/B test requires comparison of one product against one or more (for example, A/B/C/D testing or Split Test) its variations with just one major difference in an element. A particular number of uses (or time) limits the process. After the passing of this limit, researcher compares the results of two variations to divide the effect of differentiating changes on results. Every user will have an experience with one or another version of the product and the researcher will measure the quality of this experience and possibly sales conversion. Within such testing, the special attention may be paid to versions in general as well as to the placing of elements, and functionality (Unbounce).

The validity of the test and its statistical significance are based on the size of users’ sample. On the dependence of researchers’ interests, it can be from 200 tries (allows to understand which variants are obviously worse, to prepare next test) to 1000 tries (allows making final decisions from the test).

A/B and Split Tests are recommended for testing of significant changes that supposed to have a major effect on the user’s experience and interest in the product. On the other hand, this procedure is helpful for understanding what changes are significant from users’ point of view. The sample of testers may be easily constructed for a web page by showing new design to a particular group of users, however, for the testing of products like controllers this procedure may be harder as users expect a particular version of the product buying it (Unbounce).

The method in comparison to others has its advantages and disadvantages:

Advantages of A/B Testing

• They are fast

• Advanced analytics can be installed and

evaluated for each variation (e.g. click

tracking, heat maps, etc.)

• Can achieve more dramatic conversion rate

lift results

• Requires less traffic

Disadvantages of A/B Testing

• More dramatic failures

• Less specific understanding of element effects (Unbounce, p.6).

Call of duty

Call of Duty is a video game that presents the classical variant of the first-person shooter. The Infinity Ward-developed this game in 2003, its publisher is Activision. After the first release producers have constructed franchise and Call of Duty become a separate universe for gamers. Technically the game is built on id Tech 3 engine, its plot varies in dependence of release, but within different theatres of World War II, the game simulates combined arms and infantry warfare.

Among features of the game in comparison to other shooters such as Medal of Honor, the Call of Duty introduces AI-controlled allies. These creatures can react to situational changes in the process of the game and support the gamer while he accomplishes missions. The outcome of such invention is a change of model from “lone wolf” to more squad-based play.

The next games in the series have been produced in cooperation with other companies, for example, Call of Duty: United Offensive (Activision production, Gray Matter Interactive/ Pi Studios development), what is typical for the model of the franchise. Later the Call of Duty become the game for Xbox 360 and PlayStation 3, therefore users now can start it on many devices.

The gameplay of CoD allows the gamer to control an infantry soldier using various World War II firearms in the context of WWII campaigns (American, British, Soviet). The game is divided into particular missions where the player passes through the series of objectives that he is required to complete to start the next mission. The option of saving during the game vary in CoD games from any time saving/loading to checkpoint system.

Other details of gameplay include two slots for a weapon and health bar showing health points, shellshock effects, iron sights and some other specific inventions by developers.

The game is available for many players at the same time that form a battlefield effect, where single players see their co-players in different ways. Allied soldiers move with the player and assisting him in defeating and passing through the mission.

The series of games today is available for Microsoft Windows, Xbox, PlayStation, and other handhelds and consoles. The settings of next games vary from modern wars to the futuristic battles. The list of developers today is up to 15 companies, but the series is owned and published by Activision. Engines for new versions are id Tech 3, IW engine, and Treyarch NGL.

The game sales have reached in 2016 over 250 mln copies (more than $15 bln revenue). Within the universe in addition to video games, today presented comic books, card games, films and sports films, action figures, and mega blocks using the characters and the plot of CoD.

The competitive world of CoD

Games of Call of Duty franchise are widely used in E-sports. That direction of application started with the release of Call of Duty 4: Modern Warfare game in 2006. The next versions of the game become very popular among e-sport gamers as well; they are such releases as Call of Duty: World of War, Ghosts, Modern Warfare 2 and 3, Black Ops I and II and others. The form of games organization is a league, the most famous one is Major League Gaming institution.

There are two forms of competition that these leagues practice, tournaments and ladders. Within ladders form, there is the next subdivision dependence of the number of players and the level of competitive intensity. Gamers can participate in team ladder (3-6 players in the team), double ladder, hardcore team ladder (3-6 players in the team), and singles ladder. Within the team tournaments, there are rule and game settings variations that lead to the division of regular and hardcore team ladders. The system of the motivation of participation in ladder matches on special competitive websites includes rewards for the gamer in a form of experience points, that become in sum an overall rank of the user in CoD world.

The other motivation to participate in competitive websites tournaments is a gaining trophy and prizes in cash. After the win in the full competition (tournament) gamer gets his trophies registered and saved in a personal profile, the money prizes deposited to the personal bank account. Average prize to the top teams in the last 6 seasons of competition in CoD organized by Full Sail University has been $2500. In the other ladders to winners are offered medals and credits registered to a personal profile.

The most popular in the competition world game of 2014 have been Call of Duty: Ghosts. Every season in tournaments was participating up to 15000 teams. Some of this tournaments have been hosted in special Arena (the tournament website for this particular game), that had a credit system of motivation. The reward for winning the tournament has been about 15-30 credits that in regular money is about $18.75 in every tournament. The money prize for gamers participating in teams divided on the equal basis between all team members.

The biggest tournament in the history of CoD was the one of 2011 (CoD: Experience 2011) related to release of Call of Duty: Modern Warfare 3. However, on the everyday basis more important become not a size of the particular tournament, but their great number around the world and the number of specific areas and cities for LAN teams, where gamers can gain substantial prizes.

The highest popularity competitive playing of CoD has in North America and Europe, in these geographical areas enough players to pass ladder and tournaments match on the daily basis. Such geographical preference gives the specific result, among winners of top tournaments there is a proportional share of American and European players with higher results than participants from other regions do. The high results in competitive gaming provide players a high popularity, they have in average 500000 followers in social media, usually also players of CoD.

The last big competition has happened in 2017 managed by Activision. The 5th Call of Duty World League Championship has been located in Florida, Orlando (9-13/08/17). The prize pool of the tournament have been $1500000 USD, in addition, $25000 USD have been addressed to tournament’s MVPs. The goal of the tournament been to determine among 32 qualified teams the World Champion of CoD: Infinite Warfare. Regional leagues have done the qualification of teams:

2 teams – CWL APAC Last Chance Qualifier;

6 teams - CWL EU Last Chance Qualifier;

8 teams - CWL NA Last Chance Qualifier;

16 teams – CWL Global Pro League.

Xbox controllers

To be successful in the competition players need the time and sources for preparation. In such context, the quality of tools for training and competitive gaming is very important, as it can provide more comfortable conditions in the process of preparation and support players internal skills in the process of competition through better response and higher ergonomic. Two main gadgets to control the game process on the Xbox in general and in the process of playing CoD are Regular and Elite Wireless Controllers for Xbox. Next will be provided short descriptions of these products that will show the difference in the level of adjustment to the player.

Xbox Wireless Controller as its seller notice in the specification has a button mapping, textured grip, 3.5 mm stereo headset jack, and support communication through Bluetooth. As well his description promises the special feel and comfort for the user, that is a result of texturing of the grip, well-developed streamlined design, custom button mapping, and longer distance of wireless communicator coverage in comparison to previous models of Xbox controllers. User allowed plugging in headsets to the controller through the stereo headset jack of the standard size (3.5mm). In addition, Bluetooth technology available on this gadget allows the user to use such controller not only with Xbox but also with tablets and Windows 10 Pcs.

Among features that may be of critical significance for professional players of CoD, there is design and custom button mapping as they lead to easier and more comfortable manipulation with objects of gameplay and virtual personage of gamer. Headsets may be used for communication with other team members within the game.

Its producers and sellers position the second model, Xbox Elite Wireless Controller as the most advanced controller in the world. Its pack includes a number of special elements improving the quality of game experience: set of 4 paddles, set of 6 thumbsticks: standard (2), tall (2), and domed (2); set of 2 D-pads: faceted and standard. Therefore, Xbox Elite has many reasons for the status of the most advanced.

First, it is an improved fit and performance of the controller. The Xbox Elite Wireless Controller model is adjustable to gamer hand size as well as a playstyle. The most valuable for professional players may be an improvement of speed, accuracy, and reach. They can get it through swapping between thumbsticks of different sizes and shapes. In the pack of a controller included D-pads that also take ergonomics and the quality of control to the higher level.

The second feature of Xbox Elite Controller is an ability to interchange paddles to use additional fingers in the game. Four slots on the body of the controller allow attaching and removing paddles without any specific tools. The paddles take the functions of control panel buttons other than thumbsticks, this way the gamer has no need to use his thumbs to press any other keys. Therefore the workload may be divided between all fingers and inputs for shooting, aiming, jumping, and other combinations of keys and actions may be assigned to different paddles and fingers. Such assignment requires a special application.

The third feature of Xbox Elite model is a hair trigger lock that allows the player to act and in particular fire in the Call of Duty context faster. The time this function save is very important for shooter-gamers. The principle that allows maintaining higher speed is a mechanical blockage of the trigger movement just after the shot. This way gamer may get ready for the next action in the shorter time. The user can be back to full-range trigger motion whenever need it by flipping the switch back, otherwise for better tuning of settings may be used the application.

Such application allows the user a limitless customization of gaming experience and very fine-tuning of setting that provides him with the highest performance in the game such as competitive Call of Duty shooters. The parameters that producer recommend to adjust are maximum and minimum values of the trigger, sensitivities of the thumbstick, and assignment of a button to particular functions. Adjusted versions of parameters may be saved in the application in the form of controller profile, such profiles instantly interchangeable within the game process with the function of Profile Switch (2 favoured profiles per game).

In general, producer and seller point that this model of controller is designed for higher performance in the game process, therefore it is recommended to professional players. Competitive gamers will especially enjoy the ability to hold the controller easily with enhanced grip and comfortably with a luxurious feel with a soft-touch finish. The useful feature of the controller is a low-friction, buttery-smooth action and minimized wear with reinforced rings around thumbsticks, and added stability and substantial feel by the rubberized diamond grip.

Methodology

From the description above it is clear that controllers have more than one difference, therefore the test in this paper will be an extended A/B/C/D test.

The basic variant of the controller is an Xbox Wireless controller, variants for comparison includes Xbox elite wireless controller with particular buttons and adjustments turn on one at a time.

B version is an elite controller with paddles, C version is with hair-trigger locks, D version is with buttons adjusted to players thumbs.

The comparison of controllers performance happens in the context of CoD competition. The main question is if an elite controller as it promises really can improve the performance of league players. Therefore, the test will be applied to players of this particular game as an e-sport while passing a number of comparable missions.

For measurement of performance is chosen KDR (Kill/death ratio), the parameter that is significantly dependent on both the player skills and the controller quality. KDR is a very important measure for players themselves as it influences their status in the community.

To avoid the variance produced by different skills of players in the test participants play a number of missions with every variant of the controller. The number of participants is 16 and the number of missions they pass with every variant of the controller is 10. The order of controllers application varies, therefore, there is no bias related to fatigue of players happening as the test takes a long time.

KDR for every participant and controller is measured and stored in the table. The statistical task is to found out if there is a statistically significant difference between variants means in KDR.

Test

The test results are presented in the table below.

Participants-Variants

A

B

C

D

1

2,6

2,8

2,7

3,2

2

2,8

2,6

2,9

3,4

3

2,5

2,6

2,8

3

4

2,4

2,5

2,6

2,9

5

2,7

2,8

2,7

3,2

6

2,4

2,6

2,5

2,9

7

2,9

2,9

3,1

3,5

8

2,3

2,5

2,7

2,6

9

2,1

2,4

2,3

2,6

10

2,5

2,5

2,6

2,8

11

2,7

2,8

2,9

3,2

12

2,4

2,6

2,5

2,8

13

2,9

3

3,2

3,1

14

2,3

2,5

2,8

2,6

15

2,6

2,7

2,9

3,2

16

2,7

2,9

2,8

3,1

Mean KDR

2,55

2,669

2,75

3,006

From the table is clearly the preference of some versions of the controller over others, however, it is important to check the statistical significance of results.

The test hypothesis H0 is the mean for the population of KDR in conditions A is equal to the mean in conditions B. H1 state the statistically significant difference between this means.

Such hypotheses have been tested for pairs A-B, A-C, and A-D. The results are below:

t-stat

-1,64257627

-2,480694692

-5,062450646

P(T