I'm frustrated because there is an obvious difference in passing code from before I stopped beta testing (pre-April 1 or so) and now that can't be explained by changes to DB coverage.

I'm further frustrated that I see the same behavior out of veteran and rookie QBs alike, so I don't really buy the play knowledge arguments unless it takes a QB 10-12 years to get past the play knowledge gap. In fact, I'd like to see a few sims where play knowledge was removed from the equation to see if that is actually making the difference.

What I'd be curious to see (and JDB can speak to the feasibility of this from a backend standpoint) is an actual side-by-side comparison of current public code and Bleeding Edge code for either select games or select leagues... Not just for 0.4.2 vs 0.4.3, but going forward.

For example... An update to MFN-1 takes my matchup against Tennessee, our rosters, rules, gameplans... And technically runs the game twice: once against the 0.4.3 code we're testing, and once against the 0.4.2 code "in the wild". That way, we can actually see how the code changes are effecting the player performances, AI decisionmaking, and game outcomes with the same rosters and gameplans.

If we wanted to keep it a little more lightweight (having a league essentially run every game twice has got to have only a little overhead, he said sarcastically), tie it into the "Game Of The Week" logic and run that one twice... See if the supposed GOTW stays that way in both code branches.

With something like this, I think us testers can give more meaningful feedback, as we can point at something and say, "Guy X was doing Y with Gameplan Z in this code, but he's doing something completely loopy in that code, contrary to what the code change seemed to imply".

Just a professional QA tester's 1/50th of a dollar.

I'm now convinced there are mathematical problems or logical problems with the QB decision code. It may be designed in a way to simulate human error, but the errors that QBs are making are unrealistic given they are supposedly professional football players. For the most part, these errors seem to be driven by artificial limitations (distance penalties) or by attributes that should not be making that much of a difference (play knowledge). Which are things that players cannot account for in their game plans.

The QB decision making is also largely masking the real problem which is that coverage is still not good.

Finally, the problems are harder to see in MFN-1 than they are in Beta-87 because MFN-1 is a long running league with lots of overgenerated players with 100s in major attributes and full play knowledge. Beta-87 is closer to what standard leagues look like.

1) https://mfn1.myfootballnow.com/watch/8107#1470547
a) VETERAN QB WITH 100% PLAY KNOWLEDGE. Throws it to the double covered WR and not the the wide open RB or the WR at the bottom of the screen that is starting to break hard inside. Also at the bottom of the screen you can still see coverage being broken by a WR that hit the end of a route and is free running.

2) https://mfn1.myfootballnow.com/watch/8107#1470550
a) FB is wide open and is never looked at by the VETERAN QB with 100 FOV. Is this because he only has 5% play knowledge? Because if so, play knowledge needs to be turned way down in the calculation because that's absurd.

3) https://mfn1.myfootballnow.com/watch/8107#1470551
a) 100 Arm, 100 Accuracy, 100 Play Knowledge. Throws this ball 5 yards behind the WR because, I assume, the pass is going to travel 10 yards in the air and get hit by the passing penalty that was imposed post-April 1 that ended up nerfing all short passes and leaving long passes just as effective.

4) https://mfn1.myfootballnow.com/watch/8107#1470555
a) CB mirrors the WR after stutter. I assume this was the code change to improve the breaking behavior of stutter. This is about 10% better. It looks better in the viewer, but this is the same behavior we have seen in the past where DBs stop covering when the ball reaches the WR and start mirroring what the WR does.

5) https://mfn1.myfootballnow.com/watch/8107#1470567
a) Bad WR selection here when the WR at the bottom of the screen is open and the RB is open.

6) https://mfn1.myfootballnow.com/watch/8107#1470568
a) See #2 above. Also TE runs out of bounds for no reason.

7) https://mfn1.myfootballnow.com/watch/8107#1470571
a) QB doesn't give this play time to develop even though he's not under pressure at all.

8) https://mfn1.myfootballnow.com/watch/8107#1470574
a) WR at the bottom of the screen is WIDE open. The QB is LOOKING AT HIM and then turns to throw at someone else. QB has 100 FOV and 100 Play Knowledge.

9) https://mfn1.myfootballnow.com/watch/8107#1470576
a) QB throws into double coverage even though the WR at the bottom of the screen is WIDE open.

10) https://mfn1.myfootballnow.com/watch/8107#1470576
a) 100 FOV QB throws into double coverage instead of to the WIDE open WR at the bottom of the screen. I can't tell you the play knowledge here because the play does not appear on the QB player card, which is another bug.

11) https://mfn1.myfootballnow.com/watch/8107#1470589
a) Richardson has 86 catching. Does play knowledge make him forget how to use his hands?

12) https://mfn1.myfootballnow.com/watch/8107#1470594
a) What is this?

13) https://mfn1.myfootballnow.com/watch/8107#1470600
a) QB STARES, STARES AT THE WIDE OPEN TE. Bad play knowledge or not, my 7 year old makes this pass with this protection. THEN the QB throws a bad pass to the WIDE OPEN RB who has to DIVE to catch it. Passing is janky. In general, WRs are having to dive TOO OFTEN to make catches.

14) https://mfn1.myfootballnow.com/watch/8107#1470614
a) Throw to one of the WIDE OPEN WRs for a TD??? Nah, let's throw into quadruple coverage because we have 100 play knowledge of this play and it is how it is supposed to work.

That's a great idea - I would be interested to see what that would look like as well. It would require figuring out how to maintain that mirror league, not sure if it's a short-term idea but might be something I can work towards.

One thing that does happen, though, is as soon as you have one decision that changes there is a butterfly effect that diverges wildly even if all future decisions are the same. I seed the RNG with a unique ID for that sim so that I can potentially replay the sim and step through the code, but my plan never works because if there is any small change in any input variables or calculations (i.e. I change the threshold for a decision that fires once near the beginning of a game by a hundredth of a percent) that one change sends the game result in a completely different direction. Still, I do think you could see some trend differences that would be interesting and it would be worthwhile to attempt.

Ok, I'll bite - I'll set everyone's 'experience' calculation to 100% for tonight's sims in Beta87, and also disable the QB from -not- checking other receivers. This will identify if the logic to decide how open a receiver is is terribly faulty, or if the decisions you are seeing is a result of the experience calculation.

Sounds scary but I would like to see ! But there is no doubt experienced elite qbs are much more valuable under this code. Same goes for the run and pass keys

Watched my last game, after the last sutter step tweak. It seemed better to me, notably the DB play in regard of the stutter step.

It looks better, but I'm not convinced it's actually better. I'll have to dig into the beta 87 games.

Observations from the first half of my game.

Setting the play experience to 100 and turning off the do not see WR code definitely exposed some things with the passing game.

* Distance penalties on throws are out of control, and maybe, not logically correct. I'm seeing as many bad SHORT and MED passes as LONG ones.
* There are too many bad or "not clean" throws where WRs have to dive. I noticed this before, but now it's really apparent.
* QBs are not reading all the WRs. Relying on FOV as a degree of vision is good, but QBs aren't actually going all the way through their read progressions anyway.

These three things when combined with the play knowledge (which should just go away) and the not see a WR code explain why QBs often throw 30-40% games.

1) https://beta87.myfootballnow.com/watch/1814#330594
a) This pass ends up several yards behind the play. This is a good example of how distance penalties are way too high.

2) https://beta87.myfootballnow.com/watch/1814#330595
a) Same thing. Distance penalty shouldn't apply here, but it wrecks the play.

3) https://beta87.myfootballnow.com/watch/1814#330604
a) WIDE OPEN WR in the middle of the field. QB throws into a bunch of bodies. QB has low FOV though.

4) https://beta87.myfootballnow.com/watch/1814#330609
a) Distance penalty.

5) https://beta87.myfootballnow.com/watch/1814#330611
a) QB doesn't see the WR at the top of the screen, but he has low FOV.

6) https://beta87.myfootballnow.com/watch/1814#330612
a) I don't now when this stutter happens, but it happens and the LB is no where to be found.

7) https://beta87.myfootballnow.com/watch/1814#330615
a) Distance penalty.

8) https://beta87.myfootballnow.com/watch/1814#330625
a) QB is not under pressure. But doesn't fail to throw the ball poorly.

9) https://beta87.myfootballnow.com/watch/1814#330637
a) I don't know how this play got broken, but it's pretty bad now.

10) https://beta87.myfootballnow.com/watch/1814#330639
a) Another bad pass on a short throw.

11) https://beta87.myfootballnow.com/watch/1814#330644
a) Distance penalty.

12) https://beta87.myfootballnow.com/watch/1814#330645
a) Another bad DIVING CATCH pass.

13) https://beta87.myfootballnow.com/watch/1814#330646
a) RB is wide open and the QB locks onto the first WR he sees.

14) https://beta87.myfootballnow.com/watch/1814#330648
a) I know this QB has an FOV of 40, but he should see the guy running the cross. The problem is he didn't even read through the WRs on this play.

15) https://beta87.myfootballnow.com/watch/1814#330649
a) Stutter or distance penalty? You decide what breaks this play.

16) https://beta87.myfootballnow.com/watch/1814#330654
a) QB throws into the only double coverage on the field.

17) https://beta87.myfootballnow.com/watch/1814#330658
a) This isn't a bad throw. TE may be more open tho.

18) https://beta87.myfootballnow.com/watch/1814#330666
a) So the Post Flags sucks now that WRs don't cut back to the middle of the field. I always knew that play was only effective because of how it glitched defenses.

More later.

--- [Accidentally copied the whole thread twice.]

19) https://beta87.myfootballnow.com/watch/1814#330673
a) FOV? Misses the wide open WR. And throws into a crowd. So here's what I don't like about FOV. FOV controls the angle of vision, great. It SHOULD NOT determine why a QB throws or where or when. In this instance, the QB is dumb as a box of rocks, but you would think he would still be smart enough to take off running.

20) https://beta87.myfootballnow.com/watch/1814#330675
a) Can this QB get AT LEAST ONE PASS that is NOT short?

21) https://beta87.myfootballnow.com/watch/1814#330676
a) Here's another one where the QB can clear see FOUR defenders around the target and still throws it.

22) https://beta87.myfootballnow.com/watch/1814#330677
a) Here's another one. Is it FOV or INT that determines the QB throwing into double coverage.

23) https://beta87.myfootballnow.com/watch/1814#330679
a) What is this?

24) https://beta87.myfootballnow.com/watch/1814#330680
a) Oh, hey, another bad pass.

25) https://beta87.myfootballnow.com/watch/1814#330684
a) Look at the WR at the bottom of the screen. Look where the QB throws the ball. Why? The QB doesn't even finish the reads.

26) https://beta87.myfootballnow.com/watch/1814#330695
a) Another bad pass on a short route.

27) https://beta87.myfootballnow.com/watch/1814#330699
a) Why?

28) https://beta87.myfootballnow.com/watch/1814#330700
a) No good throwing options, but at least the bottom WR is single covered.

29) https://beta87.myfootballnow.com/watch/1814#330704
a) WR at the top is single covered. QB threw into double coverage.

30) https://beta87.myfootballnow.com/watch/1814#330710
a) TE is WIDE open for a first down. Let's throw into double coverage again.

31) https://beta87.myfootballnow.com/watch/1814#330714
a) Bad route for a WR with 100 route. And then 100 Catch for the drop.

32) https://beta87.myfootballnow.com/watch/1814#330717
a) Oh, look, a throwaway into double coverage. Just like December code when we pointed out there was no more throwaways.

33) https://beta87.myfootballnow.com/watch/1814#330722
a) Distance penalty.

34) https://beta87.myfootballnow.com/watch/1814#330726
a) The WR gets wide open. But the QB throws the ball too early into double coverage. QBs may be throwing too quickly still.

35) https://beta87.myfootballnow.com/watch/1814#330728
a) HE'S WIDE OPEN. WHY CAN'T THE QB MAKE THIS THROW.

OK, I've made my point. I'm done with film for this game. I might watch a few more, but it's obvious what's going wrong.

So, here's my question. There is an artificial cap on pass based on distance to reduce long pass success. I think there's a huge area to study and come up with why and how to fix it, because remember, these are not physical players occupying real space, so the interactions are very complex to make decisions about whether to catch the ball or not, and honestly I expect this to be something that really never gets away from needing to be tweaked. So while attempting to improve WR/DB interactions I'm adding boundaries to prevent the passing game from becoming too weak or too strong. So would you rather I (a) keep those boundaries to maintain realistic stats, tweaking those boundaries during each engine iteration as play logic changes; (b) remove the boundaries and just let the game go all Madden until it gets resolved; or (c) keep tweaking 0.4.3 until it gets fixed (which, frankly, probably means several calendar years)?