Scoring, cut score, curve?

[busbeepbeep]

I know this whole process in shrouded in mystery, but I was wondering what the process is.

Do you think they consider at all the number of test takers in a given discipline and "curve" the cut score based off of that? 100 engineers applied, this state needs 50 PEs this year, etc. Does the score of one test taker affect the other? Or do they convene and make their magic number, then go grade the tests with a pass/fail.

Reason I ask is I know someone taking the test, and he had a secretive attitude about studying almost like we were competing, and if he helped me do well, it would hurt his chances of passing. I think he's wrong, and it's both of us against the test, not based off of each other's performance.

Any insight?

 

[ODB_PE]

I know this whole process in shrouded in mystery, but I was wondering what the process is.
Do you think they consider at all the number of test takers in a given discipline and "curve" the cut score based off of that? 100 engineers applied, this state needs 50 PEs this year, etc. Does the score of one test taker affect the other? Or do they convene and make their magic number, then go grade the tests with a pass/fail.

Reason I ask is I know someone taking the test, and he had a secretive attitude about studying almost like we were competing, and if he helped me do well, it would hurt his chances of passing. I think he's wrong, and it's both of us against the test, not based off of each other's performance.

Any insight?

You're correct - looking at the pass rates the only thing they could do is curve up to get more people to pass - which of course meets with criticism because you're lowering the bar. In my understanding, if you get 70% or better on the questions they use, you're in. I understand they typically eliminate a couple of questions following analysis (but this typically benefits all, because a question that is missed by most is typically a bad question to begin with)

Your acquaintance is being silly.

just my opinion

 

[Road Guy]

I swore that once I passed I wouldnt care about the cut score but I would still like to know how they arrive at it..............................................................................

................................................................................

.

................................................................................

.

................................................................................

.

................................................................................

.

................................................................................

.

................................................................................

.

......................

 

[DVINNY]

This question is a few days too early isn't it?

LOL.

 

[Dleg]

I've written a few tests (simple stuff for work, like percolation tester's exams) and graded them recently, and thought about how NCEES might go about determining their cut-scores. I think I have a better idea now that I have gone through the process. If you look at how many test takers got a question right or wrong, you can start to then go back and evaluate if it was a fair question, or if there was some flaw in the question, and make a decision whether or not it was something that a "minimally competent" engineer should have known and should be counted in the final score. It kind of makes sense, I would say, when go through the process yourself, but it's really tough to explain because it comes down to a personal judgment thing.

Of course I have no idea, but I am now guessing that that is what NCEES does, but they probably have some more sohpisticated statistical methods of first flagging the trouble questions, and then the test-evaluators all have to get together and discuss, and the final decisions are probably based just on their personal judgment in the end.

And if you didn't understand that, then maybe that means I have finally hit on the answer!

 

[benbo]

I think it's been discussed before, but in addition to other methods, NCEES uses a method called equating to check their cut scores. This means, as est as I can understand, that there is a group of questions for which NCEES has very good statistics on how many "passing" examinees will answer correctly. Say they have a group of 100 questions that they figure 50% of all examinees will answer correctly, and they check this over time.

They continually repeat a few of these questions on each exam. Like they might select 5 of these "known percentage" questions to put on the test. Then, they can guage whether the problem with a test is the test difficulty or the people taking the test. Say they assume that people will get 60% correct on average. I'm not exactly sure how they do it, but this is part of the process. You can find equating explained a lot better on the internet.

This is one reason they don't want people to say what's on the test - because they repeat problems.

 

[busbeepbeep]

thanks for the insight guys....of course I don't have to worry about this right now...just gotta give em hell tomorrow and score above their magic number.

 

[Guest]

52.4444

 

[GTScott]

Are the results out yet? Does anyone know what date/hour they post them? Is it true that the cut score is 7?

I feel for you folks waiting for results. I did not think anything was as bad as the actual exam last year until I experienced the dreaded wait. Then, to make matters worse, RG e-mails me and lets me know that GA has posted results but proceeds to take a 4 hour lunch break right before getting to my name.

 

[Guest]

Are the results out yet? Does anyone know what date/hour they post them? Is it true that the cut score is 7?

Seven ?? :huh:

I am thinking you are aiming a tad 'low' with that guess :joke:

JR

 

[GTScott]

Fine, the cut score is really 9.

 

[jfusilloPE]

I honestly believe that there is no cut score or well defined operation as to how they score the exam. Especially since NCEES and many States have gone away fom giving a "score".

I think someone takes all of the tests into a room, they then proceed to throw them into the air. Depending upon whether your score sheet lands in one pile or another or face up/face down and they pick the smaller pile and those are the ones that pass.

Just my two cents.

 

[testee]

well then I hope that I "score"!

 

[Guest]

I think someone takes all of the tests into a room, they then proceed to throw them into the air. Depending upon whether your score sheet lands in one pile or another or face up/face down and they pick the smaller pile and those are the ones that pass.

Don't forget the compiled information they keep about how many exams you have taken as well as how much you have paid out-of-pocket for exam items through *cough* "other intermediaries" for publications, references, and exam prep classes.

I am sure that examinee-cost data balanced against benefits to :Chris: are factored and weighted into the overall calculation of the final score as well. That is why it is such a tightly held secret - would you want to tip off the suckers that are paying you off ?? I think not! It is a pretty good scam if you ask me. :2cents:

Having said that, I have often wondered what the cumulative probability distribution score for passing looks like.

I have thought about developing a rudimentary regression analysis based on a survey of exam-takers, but I would rather stick to my conjecture - it sounds better! :happy:

JR

 

[Hill William]

I have thought about developing a rudimentary regression analysis based on a survey of exam-takers, but I would rather stick to my conjecture - it sounds better! :happy:
JR

 

[jfusilloPE]

^^^My sentiments exactly