More About Beta-Testing

Reading through the author’s forum from the recent comp, I ran across some discussion of how mean/unfair reviewers can be; this is no doubt true, though it is also true that people get irked about having their time wasted by a game from an author who didn’t do (what they consider to be) due diligence in writing the thing. It’s a fine balance. (And no, the purpose of the comp is not to encourage newbies. Not as such. It may do that, which is fine, but that’s not the mission of the competition. I tend to think that it’s about producing more cool IF, and in a context where that IF gets noticed and talked about, both inside and outside the community.)

Anyway, the particular bit that caught my attention was this:

Certain reviewers seem to imagine that hoardes of willing beta-testers drop out of the sky at the slightest mention that you’re working on a game.

I kinda wonder how much time these reviewers have spent beta-testing people’s games?

Harry Wilson aka Conrad Cook

Actually, lots of the reviewers have put in significant testing time. This is part of what makes them so sensitive to all the ways in which a game can go wrong. Beta-testing is itself a lesson in game design and implementation: you get an up-close view of what can go wrong and how someone else goes about fixing it. But second: if you are having a hard time finding testers, ask people. You can post a request on RAIF or join the IF betatesting site, but if that is not producing enough feedback, go on ifMUD and find folks; or approach people individually by email and say, “Hello, would you please test my game?”. It helps to include some basic information about what kind of game it is — and, perhaps, an explanation of why you think that person would be a good fit and/or might enjoy doing it. (This latter part is optional, but it might help if you’re addressing someone you haven’t worked with/talked to before at all.)

Sometimes people will be too busy, and will say no. But it’s not wrong to ask. People are much, much more likely to say yes if you ask them specifically and personally than if you put out a general call for volunteers. (Later on when you’ve written some games that people know are good, putting out a general call for volunteers may turn up more testers — but by that point you may already have a team of people you like working with, and not need to do that so much anyway.) Some authors have a rule of seeking as beta-testers the very reviewers that were most harsh on them last time, but that’s certainly not obligatory (and heaven knows you may find it’s more to the point to work with a beta-tester who is sympathetic to your basic vision but has high standards of craft).

Finally, the community expectation in general is that beta-testers will not diss your game in public, will not disclose the errors they found, and will not distribute a pre-release copy; and conversely that you will credit your beta-testers in some way in the finished game. (An ABOUT or CREDITS verb is usually used for this purpose.)

But when all is said and done, it is not unreasonable for players to expect that the game you offer them has been through testing. If you haven’t tested it, you haven’t finished it. If it’s not finished, don’t submit it. It’s that simple. Seriously.

28 thoughts on “More About Beta-Testing”

  1. I haven’t looked back at the forum and to my knowledge nothing juicy or whatever was said by me or anyone else. BUT, Conrad’s e-mail inviting the authors said:

    “…it is against the rules for us, as authors of this year’s IFComp, to talk in public about our own or others’ entries…

    However, *private* discussion is permitted, and discussion on a private forum is permitted.

    Thus, Rule 5, the IF Comp Author’s Club — a place where we can talk shop and comment on the judges’ commentaries privately…”

    He didn’t say anything about how he was going to make it public afterwards. Even if the contents are innocuous, am I weird to think this is bad form?

  2. Hm. Yeah, maybe — I assumed the situation was otherwise, since the forum currently has this notice at the top:

    Hello, authors of IFComp 08!

    Think of this as a smoke-filled room where we can discuss our games as authors while the competition is in progress.

    As members of this forum, you are implicitly bound by rule #5 of the IFComp, by your honor as IF Comp contestants, and by the moral integrity which inheres to all writers of IF, not to disclose the ultra-secret discussions held here, until IFComp votes are cast.

    When the competition is done and the ban on author discussion is over, this forum will be made public.

    …which I assumed had been there all along.

  3. Despite the fact that I of course read the Rule 5 forum as soon as I was able, part of me was disappointed that the forum was opened to non-authors. In the end I don’t know if such a secret club is a good idea really, but it felt kind of cool when it was created.

  4. Emily Short wrote:
    And no, the purpose of the comp is not to encourage newbies. Not as such. It may do that, which is fine, but that’s not the mission of the competition.

    Maybe not, but according to this RGIF post, Gerry Kevin Wilson, the creator and first organizer of the IF Comp, apparently thought it was the purpose:

    […]
    While I enjoy every single game I’ve played from the competition, the unpolished games are my favorite, because they represent someone new entering the field.
    […]
    Let us hope that everyone learns that “This game sucks.” reviews help nobody, least of all the game’s author.

    A particularly hurtful remark can rob us all of a Leon Lin or Andrew Plotkin. Criticize, but speak constructively and kindly. The contest is for beginning
    authors, and they are often uncertain and easily hurt.
    If you cannot do these
    things, then I care very little what your opinions on the entries were.

  5. “New author” is not synonymous with “bad author”. There are multiple examples of first-time authors scoring near the top (or at the top) in various IF competitions. And there are several notorious examples of seasoned authors who place in the bottom 10% of the comp, year after year. Authors, whether seasoned or not, need to set high standards for themselves.

    I haven’t found it that difficult to recruit beta testers from rec.games.int-fiction. You need to write a good blurb for your game to recruit testers.

  6. So what’s the best place to sign up for betatesting if games?

    The IF Beta-testing site is still active; there’s some general testing advice on the front page, and information for both authors and testers. If you sign up to be a tester, you will periodically receive email about new games available to test. If you have spare time at that moment, you can take them on; if not, just ignore and wait for a request to come through when you have a little more freedom available.

    Gerry Kevin Wilson, the creator and first organizer of the IF Comp, apparently thought it was the purpose

    Hrm. Well, okay; that’s not the reason for its existence that I remember being articulated at the time, but I can’t really argue with your evidence! On the other hand, in practice the comp has not existed primarily as a newbie-fostering institution, even if it does that sometimes.

    For me personally there are several different things I keep in mind when writing reviews, which pull in contrary directions:

    1) I’m an author, I’ve gotten some unpleasant reviews, and I empathize with the situation of other authors. I don’t want to crush anyone’s spirit.

    2) It’s condescending to praise something that isn’t praiseworthy just because you feel sorry for the author. There’s kindness and then there’s respect, and if I had to pick one for myself, I’d pick respect every time. If it’s impossible to combine them, I try to show respect.

    3) Despite my attempts to write empathetically, I don’t know most of these authors personally (especially not the newbies) and therefore I can’t really judge how they’ll react to what I say. Second-guessing too much is pointless.

    4) The reviews I write also get read by players deciding what to play, and I feel some obligation not to mislead them about the quality of the work.

    5) Reviews that are too sanitized are boring to read and not that informative to anyone. There is no purpose in writing reviews if they are all going to be delicately-balanced point-and-counterpoint thingies.

    6) There is *some* behavior (like not testing a game!) that I think it is a good idea for the community to discourage. On the one hand, yeah, we want to help and encourage novice authors so that they become better and stick around. On the other, if there’s something glaringly problematic about their process and it isn’t pointed out, they *won’t* become better.

    7) (This is where I will sound most cold-hearted.) Yeah, novice authors are sometimes uncertain and easily hurt. Artists have a thin skin sometimes. To go with that, though, they also need a certain amount of self-belief, of vision, of drive. IF is a huge amount of work to write, and for very little reward. If you don’t have some internal compulsion to do it, you probably just won’t.

    Anyway, there is no game so good that it will get good reviews from everyone. Someone will hate it, sooner or later. It’s safer and saner to set a different goal — like hoping that someone(s) will really *get* your game, even if not everyone likes it.

  7. Thank you for your answer, Emily!

    I didn’t quote Gerry Kevin Wilson’s post because I fully agreed with it (I didn’t); I’m sorry if I didn’t make it clear! I quoted him because I supposed he knew what the purpose/mission of the competition originally was, at least the way he saw it. But the comp very quickly became much bigger and much more successful than he had imagined, so it’s not surprising that there are many other purposes now…

    I agree with you, actually: it’s certainly neither possible nor desirable to always write only nice and encouraging comments in reviews. Being kind to newbies who created flawed games with good ideas, or at least visible work invested, is one thing; being kind to authors who seemed to make no serious effort to write a good game is another. (And of course, there’s no reason at all why we should write good reviews of intentionally bad games.) And perpetually nice and enthusiastic reviews do tend to resemble advertisements, which isn’t a good thing.

  8. While I’m sure I read the rules of the private board, I wasn’t fully aware it was to be made public. Since there was that niggling thought in the back of my head, I must have read it and absorbed it without consciously thinking it was true.

    But I re-read some of the threads just now and I’m glad I didn’t say anything too embarrassing.

    This kind of reminds me of the fairly recent controversy over the release of the INFOCOM drive, and the e-mails that were made public, when at the time they were written and read, there was no expectation that they ever would be made public.

    For historic purposes, I am grateful the contents of the drive were released – it gives me greater insight into the operation of one of my favorite companies, and some of my personal heroes, none of which were in the slightest diminished in my eyes after reading that great thread about “Milliways”.

    I’m glad I wrote positive things, and gave out positive advice and opinion, though. Because in a truly private forum, I can be a bit of a dick.

    :-)

    And thanks again, Emily, for not one but TWO solidly good reviews of Piracy 2.0. I really appreciated the first, but the second hit home even more because I felt it was even more positive, and written after the competition when you certainly didn’t have to.

    Thanks!

    Sean.

  9. I didn’t have you in mind when I wrote that, Emily. Actually I was bent out of shape at someone who wrote a review that swore at me: even apart from that this guy was pretty malicious. And I got the sense (possibly wrong) that he had made pretty limited contributions to the community.

    I’ll put it this way: if reviewers are free to gripe about our games, I think you’ve got to acknowledge it’s fair for game authors to retro-gripe about reviewers.

    If there’s a standard regarding who the IF Comp is “for,” then that needs to be made officially explicit and there should be a qualifying round. As things stand, not even someone with your stature gets to tell us we can’t or shouldn’t compete.

    For my part, I learned more about game design in that two-month span than I have in the past two years. So, to be honest, I’d do it again — swearing reviewer and all.

    Conrad.

    ps – I think some of the bad feeling comes from the expectation judges have that they play *all* the games. I think that’s an unreasonable self-imposed standard. I didn’t have that standard; games I didn’t like I skipped. And some of those were quite well-made games that just didn’t suit me.

    So, no hard feelings, Emily — but no. I’d do it again.

    C.

  10. That’s a point I’m glad to stand corrected on. And for my part, I think you write model game reviews.

    Let me ask: Who are you writing this blog post for? What is the purpose of this communication?

    If you’re writing the ’08 newbies like me, I think we pretty much learned our lessons. If you’re writing the newbies of ’09… most likely, they’re not here.

    I’d say the better strategy would be to see if the IF Comp guys would put the beta-tester link on their homepage and in their call for games. Knowing about it would have made my search for beta-testers easier, and I expect that’s true of a lot of newbies.

    Or, to write and strategically repost a guide for newbies who are entering a comp. Like a netiquette guide or a FAQ. In fact, I want to get the Rule 5 forum going early next year, and I see showing newbies the ropes a primary purpose of it.

    The thing I’m trying to get across is, beta-testing doesn’t cure all ills: because when you’re first getting the hang of a skill, you can identify problems without really knowing what to do about them; and your tastes probably aren’t yet refined enough that you can identify problems with specificity.

    So… what I find troublesome about the perception that new authors shouldn’t participate in the IF Comp is that I think there’s a greater need for a comp that newbies can participate in than one only for pros. Because comps promote newbies into game designers. Whereas pros know how to promote themselves.

    Going through the archives this Comp, I noticed that a year or two ago someone complained about you being in the comp, because, she felt, it meant nobody else could compete. I think that was unfair and unfortunate, and as I recall she later backed off — I’d hate to think something like that would discourage you from writing games, or entering them.

    The purposes are different, though: for a newbie, the comp tests whether they understand how to put together a basically-passable, fun game. A pro uses the comp to push the boundaries of what has been done.

    If you make the Comp officially not for newbies — it seems to me you get the old Catch-22, where nobody will hire you if you don’t have experience.

    Conrad.

  11. This is the thing about CyberCow: depending on your interpreter, operating system, state of mind, external temperature, and I think the position of Mercury, you could wind up playing any one of five or so different versions of the game, varying only in levels of brokenness. I am still not sure I ever played the working version, and Riff (whom I am assuming is Negative Swearing Bastard here) played one so broken that VERB NOUN failed to work even though the hints said “There is a NOUN you need to VERB,” which is, I submit, legitimately frustrating, something that might cause a Negative Swearing Bastard to carpet-F-bomb the tri-state area. (I think he went back and added the word “game” to clarify that he wasn’t swearing at the author, but the game, in exactly the same way he swears at the wireless internet and other broken things, loudly and vehemently.)

    No matter how good your game is, if the interpreter renders it nearly unplayable, it will be indistinguishable from a bad game. CyberCow simply did not work through Spatterlight. Some Mac users figured out they were not playing the game as intended and refrained from scoring it, others assumed it was another underimplemented untested entry and reviewed it accordingly. People aren’t normally psychic.

    I don’t know, I could go on for paragraphs about competition reviews and the relative merits of niceness and honesty and all sorts of crap, but that almost isn’t even the issue. Your reviewers are going to be as tactful or as blunt as they choose to be. They just are. That is how people work. One person might say “The shimmering perfection of this game was marred by a particularly egregious guess-the-verb issue in the kitchen scene,” while another might say “What the profanity? What the profanity profanity am I supposed to do with this profanity thing in the kitchen? Profanity… I’m’a look it up. OH YOU PROFANITY I TRIED THAT! THIS PROFANITY GAME IS PROFANITY!” No matter how nice people are about the guess-the-verb issue in the kitchen scene, it’s just better if there isn’t one, and you the hypothetical author have control over that; also “profanity” no longer looks like a word.

    I don’t mean to imply that a submitted game needs to be perfect. People tend to be willing to forgive a few bad things surrounded by good things far more easily than they will even notice a couple good things in a sea of crap. All you have to do to get a good review from a fair reviewer is to write a good game, which one would think you’d want to do anyway, just for yourself. I don’t know. I go on about people and their tendencies, but will never actually understand them.

  12. Let me ask: Who are you writing this blog post for? What is the purpose of this communication?

    I was writing largely in response to the perception that beta-testing is somehow an unfair thing to expect, or that there aren’t resources for doing it, whether or not you personally still think that.

    If in fact the problem is that comp authors were trying to find testers and failing, then we need some better communication about how to recruit them, and this was meant to be a gesture in that direction.

    I also think that asking people directly is the best way to get at least one really dedicated tester: someone who not only sends you a transcript or two, but who likes your game and feels a certain amount of commitment to helping you get it off the ground. You don’t need all your testers to be like this, but having one or two on side can improve your work immensely. That person is more likely to be willing to play your game multiple times and to comment on how it’s improving. He or she is also more likely be willing to make large criticisms, like “you know, this part of the story line doesn’t work” or “I really think you need to rethink how you’re handling the magic system” or whatever. Sam Kabo Ashwell was that person for City of Secrets; Admiral Jota for Savoir-Faire; other people have played that role on other work of mine. I’d be lost without it.

    If you’re writing the ‘08 newbies like me, I think we pretty much learned our lessons. If you’re writing the newbies of ‘09… most likely, they’re not here.

    Undoubtedly some aren’t. Just as likely, some are. Before I wrote my first published IF game I spent about two years lurking in the IF community, observing and reading articles. Other people have indicated that they do the same, and several people have told me that they read a number of my articles/posts before writing their first games. So I don’t think it’s unreasonable to imagine that I might be reaching someone who still means to write something someday.

    At any rate, this isn’t a punitive exercise; I’m not trying to make sure that anyone has “learned their lesson”, as such.

    I’d say the better strategy would be to see if the IF Comp guys would put the beta-tester link on their homepage and in their call for games.

    That’s something you could address with Stephen Granade if you want. I thought I recalled there being such information the last time I signed up, but I could be wrong.

    The thing I’m trying to get across is, beta-testing doesn’t cure all ills: because when you’re first getting the hang of a skill, you can identify problems without really knowing what to do about them; and your tastes probably aren’t yet refined enough that you can identify problems with specificity.

    Possibly not. There are some other ways to try to address that: read a lot of reviews; test for other people; discuss craft issues regularly; come to RAIF or the intfiction forum or ifMUD and say “I’m having (vaguely described) problem: how would you address it?”.

    The betatesting method (testing for others, I mean) is particularly useful and underused, I think. Seeing how someone else attacks problems in their work is a fabulous lesson in craft. (Beta-testing Dreamhold was especially fun: zarf is really good at the tactics of design, directing the player’s attention towards the right things to get a puzzle solved, and watching him refine the puzzles was nifty.)

    So… what I find troublesome about the perception that new authors shouldn’t participate in the IF Comp

    But I didn’t say that! I didn’t say anything like that! I don’t think I’ve seen many other people saying that either. (Maybe someone has somewhere — someone or other will always say rude stuff that deserves to be ignored. But this is hardly a majority opinion.)

    What I said was that I don’t think the Comp is for encouraging newbies: I do not consider that to be its primary purpose. Newbies are obviously welcome to enter, and I’m glad to see it when they do. Nor do I presume that a game from a new author is going to be inferior. Violet and Edifice were both comp-winning games from brand-new authors, and Star Foster hadn’t written any other IF before collaborating on Slouching Towards Bedlam. A number of other games I look back on fondly were also first-time entries: Gourmet, The Chinese Room, Moonlit Tower, Rameses… there’s a long list. Some of those authors went on to write other excellent work.

    But if the Comp existed for the purpose of encouraging newbies, then (a) the reviewers would, I think, have more of an obligation to lower their expectations and write in a more teaching/encouraging mode; and (b) experienced authors would have an obligation to stay out of the pool. And I don’t see either of those things as desirable.

    Because comps promote newbies into game designers. Whereas pros know how to promote themselves.

    Nah. Some people have written great first games; some have become well-known for stuff they released outside the comp; some have entered the comp multiple times for years and never gotten any better; and no one here is a “pro”. (Howard Sherman might have a claim, but he hasn’t entered the comp in years, and his last submission was not in my opinion especially great.) Even for those of us who have written many IF games, the process is still difficult, it’s still possible to get nervous about how things are going, and it’s still possible to release a non-competition game to less feedback than we might have hoped for. I’d say it has gotten harder, not easier, to write IF since I started.

    Going through the archives this Comp, I noticed that a year or two ago someone complained about you being in the comp, because, she felt, it meant nobody else could compete. I think that was unfair and unfortunate, and as I recall she later backed off — I’d hate to think something like that would discourage you from writing games, or entering them.

    There were multiple people who expressed that opinion, and yes, it was part of what I was reacting to in saying that I don’t think the Comp should be specifically a newbie-fostering institution. I think there are several good reasons why we want to keep the Comp open to everyone:

    1) it’s one time of year when IF gets a certain amount of attention from more mainstream gaming communities, and that is in part because there are usually some really good entries. While there would still be some really good entries in a newbie-only competition, the proportion would probably go down.

    2) if the overall quality drops, reviewers are also less likely to be attracted to playing all the games, and then the feedback for which novices enter in the first place will go away.

    The purposes are different, though: for a newbie, the comp tests whether they understand how to put together a basically-passable, fun game. A pro uses the comp to push the boundaries of what has been done.

    I think it is possible to know (yes, even for a novice) whether you’ve written something “passable and fun”. Play it. How does it compare with other good games of your experience? If you don’t know, play more IF to give yourself background, and ask your beta-testers directly. If your beta-testers can’t or won’t tell you, get new ones. (See suggestions above.)

    You probably can’t judge whether it will be fun for everyone (different people like different things), but you can probably tell roughly where it falls relative to the existing body of IF.

    And definitely some judges do not want to be participating in a writing workshop, so it annoys them when games are entered where the author has not asked himself whether the game is good enough to be worth the investment of between 5 and 120 minutes. Like you, I’m willing to put a game down quickly if it is terrible, so there’s a limit to how much of my time gets wasted. Some people feel more dedicated than that.

    There’s a second frustrating thing, which to my mind is even more painful: playing a game that could have been really good with another month or two of work. Its mediocre placement pretty much guarantees that it is never going to get that work, so the author’s time so far has been wasted, the idea is shot, and the author may go away discouraged and sad when he could have had a XYZZY-worthy piece of work if he’d just held it a year or started sooner or entered it in Spring Thing instead.

    Even newbies don’t have to use the Comp as a writing workshop. There are better ways of workshopping a game.

  13. The reason we swear at people, or things, is to attack them. There might be exceptions to that, but the review in question certainly isn’t one of them.

    And that’s fine: when you put yourself on a stage it’s part of the deal that people might throw rotten tomatoes at you.

    In that case, I don’t think the reviewer wanted to do anything but write a negative review: I’m lead to that conclusion, for example, by the fact that he failed to respond when I pointed out he had corrupted the game file by converting it.

    So — I didn’t like reading it. It was an effective attack (as I’m sure he’s glad to know). I won’t say he shouldn’t have done it, because the game he got out of the box was badly damaged, and because I don’t argue against people’s natures.

    But I certainly think that if he’s free to swear at me, directly or indirectly, then I’m free to gripe about it.

    Conrad.

  14. I think it is possible to know (yes, even for a novice) whether you’ve written something “passable and fun”. Play it. How does it compare with other good games of your experience…

    Well, sure. But by those standards, I thought LAIR would score in the 5.5-6.8 range. *I* had no trouble running it; it was winnable; it *did* go through beta-testing, which identified some problems, but which didn’t identify the interpreter problem, because I distributed the interpreter with it; I concluded that LAIR was a bit thin, but playable, and people might dig it.

    [SPOILERS for LAIR of the CyberCow]

    But besides the technical problems — which amounted to an unidentified assumption — there were cultural problems. People didn’t know that you leave milk for fairies. I thought everybody knew that. People knew, technically, but weren’t present to the fact that milk comes from cows.

    I had one guy who was so kind as to write me — and this was very illuminating for me — and say that, if he was supposed to milk the cow, I should have mentioned in the description “that the cow has breasts.”

    Well, I grew up on a farm: a cow has udders. Or tits. Humans — men and women — have breasts. And the purpose of having cows is their milk. Or beef.

    I know intellectually that most people didn’t grow up on farms, and for example that most people don’t know how to milk a cow; but that people wouldn’t make the connection just didn’t occur to me.

    So, there were cultural assumptions I was making, mtoo, which resulted in the game being poorly-clued. ore beta-testing *would* have caught that, and I regret that I didn’t know about the beta-testing site, etc.

    As it happens, it was an incoherent mess to most people. Two reviewers, that I know of, just happened to run it on the proper interpreter, and just happened to have the right assumptions: and both said, basically, “Eh. It’s okay.” (And I gather that it was the few judges like them who gave the game its 5’s and 6’s.)

    –To tie up a conversational thread, Emily, I didn’t mean in the post you quoted to say that beta-testing wasn’t desirable or shouldn’t be done.

    Always good talking…

    Conrad.

  15. I was the one who failed to respond when you pointed out I had corrupted the file by converting it, mainly because I never got the converted version to work, and it seemed easier to go across the room and play what I thought at the time was the actual functional version but was actually the broken Spatterlight version, which was far less broken than the one I had played and therefore deceptively fixed-looking, and all of this sounded far too convoluted and brain-hurty to get into. Maybe I should have.

    I responded here not to tell you what you could or couldn’t gripe about, which I would never dream of doing, but because it seemed as though you, a fellow human being, were hurt in the feelings area over what was essentially a public crankiness at a parser. (We were both reviewing as we played, not after the fact, so all of the visceral anger reactions were lovingly preserved.) Despite the evidence, I genuinely dislike it when fellow human beings get hurt in their feelings place, so I thought I’d pop in and say “that thing you have been taking personally was not intended as an attack, and you do not have to feel bad about it.” If your response is to argue that yes, it was intended as an attack, and you do have to take it personally and feel bad about it, then there is not much I can say unless I resort to sarcasm, in which case you’re completely right, there is only one reason to swear at anything ever, and we sit around in our underwear all day trying to think of new ways to be mean to that Harry Wilson person because my God do we ever hate him. (This is clearly untrue. We hate everybody and sometimes put on pants.)

  16. If your response is to argue that yes, it was intended as an attack, and you do have to take it personally and feel bad about it, then there is not much I can say unless I resort to sarcasm,

    Jenni, I find it remarkable that you’re the one troubling to have this conversation, when you aren’t the one who did the swearing.

    In my view, grown-up people are answerable for their own behaviors: and they don’t hide behind their friends’ skirts.

    Conrad.

  17. In my view, grown-up people are answerable for their own behaviors: and they don’t hide behind their friends’ skirts.

    This subthread has, I think, reached a point where it doesn’t belong here — if you want to continue to discuss, please do so in another forum/email; followups here will be moderated. Thanks.

  18. At the beginning of this thread, I thought I had some ideas on how to address both the newbie question and ultimately produce better comp games. Thing is, the consensus is that many of the games would have been improved with more testing/feedback.

    One of the drawbacks in the IF world is that there isn’t a well-known counterpart to a writer’s circle, which is often a valuable tool in the literary world.

    If there’s something like that in the IF world, I haven’t heard of it. The newsgroups certainly aren’t it.

    Perhaps it would be best after all to set up a completely private (before and after) forum for just the writers, with a section reserved for first-time entrants, and make it available upon intent to enter. Constructive dialogue rarely happens without a sense of safety, and I’m not sure how an eventually-public forum fosters a sense of creative safety.

    I’d say there should be a newbie comp as well (separately), except I doubt it would get many entries. As many have stated before, “the” comp has the best exposure out of any of the contests. If there was such a beast that was only open to unpublished authors, the judges should be drawn from volunteers who have been involved in IF somehow, and the part of the judging duties should require a review of some kind that focuses on how a game could be improved.

    Basically what I’m getting at here is a comp where the focus is on how the game could be improved. Perhaps “showcase” is a better word. Perhaps every year, there could be a “best of last year” section where authors could showcase how the games have changed. This could be an incredibly valuable resource not just in the sense of directly resulting in better games, but also by revealing the actual process of how games improve (or don’t) to aspiring writers.

    In a similar vein, for several years now I have really wanted to see a “most improved game” special category in the comp. Open it only to entrants from previous years who scored below an arbitrarily chosen number, who resubmit games they’ve reworked/tried to iron out. Give them a chance to redeam themselves in their own eyes and those of their peers.

    Sure they can just go make a new game with the knowledge they’ve gained, but honestly a number of the borderline or “bad” or unfinished games (this year especially) had neat ideas that are worth expanding. Even the meanest reviews usually had a “this would be neat, if only…” tone to them.

    As for the reviews, well. I have to admit that I laughed my butt off at some of the ones I would find most insulting if it were my game. If I’m honest that’s partially a bit of schadenfreude. But more legitimately, this is exactly how I was feeling at the time in a number of cases. I was a monkey with a bone, banging on a monolith that someone else spent a lot of care making (even on the lower-scoring ones, and probably especially if it was their first game).

    But even though I enjoyed reading them, I don’t think they were a good idea. I7 has brought a lot of new people in with a lot of potentially neat ideas. Scaring them away or them becoming resentful of the community is bad in the long run. I say this even though a couple of the reviewers are good friends of mine, who have as much right to say exactly as they think as I do to say “I don’t know if that was a good idea in the long run for the IF world.”

    I just hope some of the writers will pick the valid complaints from the words that surround them and go on to make better stuff.

  19. Also, I apologize for the general lack of grammar in my previous comment. I’m at the low energy end of an all-nighter.

  20. One of the drawbacks in the IF world is that there isn’t a well-known counterpart to a writer’s circle, which is often a valuable tool in the literary world.

    If there’s something like that in the IF world, I haven’t heard of it. The newsgroups certainly aren’t it.

    There have been some forays in this direction. Jim Aikin ran an IF workshop earlier this year, and I believe that some of the games in this competition came through it. It may be that he or someone else will want to do that again in the future; I don’t know.

    ifMUD also functions, not exactly as an official workshop, but as a place where craft ideas can be discussed, and as a launching point for more formalized kinds of collaborations and feedback groups. But one usually has to actively pursue these things.

    In a similar vein, for several years now I have really wanted to see a “most improved game” special category in the comp. Open it only to entrants from previous years who scored below an arbitrarily chosen number, who resubmit games they’ve reworked/tried to iron out. Give them a chance to redeam themselves in their own eyes and those of their peers.

    Yeah, I can see where you’re going with this; at the same time I can see some potential issues. The arbitrarily chosen number would likely displease some people; not everyone wants to re-judge games that they’ve already played once; etc. I suspect that it would be felt to be a bad fit with the existing comp.

    I do agree, though, that it would be great for some of the not-quite-right games to be redone and republished; and this applies not only to low-scorers but even to high-placing games that could have been better still. The trick is providing more incentive to do that. Some authors are sufficiently compulsive that they do post-comp rereleases (which is great) — Eric Eve, for instance, has been consistent about issuing post-competition versions of his work that address things that came up in the comp.

    I feel a little guilty on this score because one of my outstanding obligations (as I see it) is to rewrite Floatpoint; but what it needs (based on the feedback) is not a couple of bug-fixes, but to be about 100-150% longer, and to incorporate the freedom to converse freely with other characters.

    This is doable, but I’ve been engrossed in developing the conversation system to do it… and then writing some other sample games to prove to myself that the conversation system works as I want it to… and so on. So one of these days I do still mean to come back to it. In the meantime, Floatpoint continues to accumulate negative reviews about the bug in the competition release. I suspect that reputation will always cling to it, even when I get around to fixing it.

    As a result, I’ve been thinking about this from the perspective “how would I feel if I had to do the same amount of work but on a game that was not very well received to start with, and if I also felt a little discouraged about my skill-set?”.

    I wonder if what we need might be a model very unlike the competition: something with no fixed deadline (so that no one will feel rushed to meet same), and something that could bring substantial new attention to a work. Maybe we need a person or group functioning as something between a publisher and a literary agent — not necessary with an agenda to make money, but someone who could provide editing feedback until the game was improved, and then help repackage the game and get it reviews. Maybe they could help put together cover art and feelies; maybe they could help the author approach websites outside the community that would be interested in running profiles, too.

    The problem is, that’s a kind of thankless job for the hypothetical editor/agent. So I don’t know whether this is practical to imagine.

    But I have a strong sense of waste seeing some of the nifty ideas not turned out to their full potential, and their authors perhaps saddened or discouraged by the experience. Imagine how much stronger this year would appear in retrospect if, say, Cry Wolf, Magic, and The Ananachronist were polished into top-tier games and rereleased.

    I’m also reminded that at some point I need to play and review Lydia’s Heart, since that’s a case where the author did take on board a lot of feedback and go through the hard process of reenvisioning and totally reimplementing his work.

Leave a reply to Doug Egan Cancel reply