October 2019 Post Exam Wait Period - Welcome to the Suck

[Orchid PE]

It might take a while for information to get into my head, but once it's in there it takes just as much effort for it to be forgotten .

 

[RBHeadge PE]

@RBHeadge PE, @jean15paul_PE So like this for an example session (but normally done for the entire exam):


Question #


Right/Wrong


Cut Weight


1


1


0.2


2


1


1.0


3


1


0.4


4


0


0.1


5


1


0.6


6


0


0.6


7


1


0.2


8


0


0.4


9


0


0.9


10


1


0.7


11


1


0.9


12


1


0.2


13


0


0.6


14


1


0.6


15


1


1.0


16


1


0.1


17


1


0.6


18


0


0.5


19


1


0.7


20


0


0.6


21


1


0.8


22


1


0.9


23


1


0.3


24


1


0.5


25


1


0.9


26


0


0.4


27


1


0.7


28


1


0.7


29


1


1.0


30


0


0.3


31


1


0.4


32


0


0.5


33


1


0.6


34


1


0.2


35


1


0.3


36


1


0.8


37


1


0.2


38


1


0.5


39


1


0.6


40


0


0.4


 


29


21.9


 


Exam Score


Cut Score

I'm too busy to do that math. But if column B adds to 29, and column C adds to 21.9, then sure it's like that. Except they would probably round up C to 22.

 

[LyceeFruit PE]

"exam specification" - What do you mean by this? Like, are all the questions in the bank a "specification" or is morning session A + afternoon session C a "specificaiton"?

So for April 2018 exams & after, they changed the exam spec for the Power PE. It's the one you see on their website today.

Prior to that, the exam spec did not include NFPA 70E, NPFA 30B, NFPA 497, NFPA 499. There are 9 categories today but prior exams had 10 categories you could be tested on. When they switched it to 9, some of those sub-categories shifted into the bigger categories and then they also added more protection & more code questions to the exam. So for the April 2018 exam, they'd have to have the cut score meeting but exams after that are under the same spec so the standard was previously set as @RBHeadge PE said.

 

[RBHeadge PE]

they determine the cut score is 68 (out of 80) based on the difficulty and expectations for each question. Once that cut score is determined, you have to get 68 right to pass. It doesn't matter if your 68 points come from easy questions or hard questions. They all count the same toward your pass/fail.

Wait how did you get the real cut score for October's power exam?!

 

[Tiger15]

So the cut score meeting determines the weighting for an entire question bank, which is some number of problems much greater than 80? Then each administration pulls from that bank to created different tests?

edit:: this was a question... mostly directed to @RBHeadge PE

 

[Tiger15]

Wait how did you get the real cut score for October's power exam?!

You would only know that was the real cut score IF YOU WERE MAFIA!!!! 

Hey, if yall are bored, check out our mafia game. 

 

[Orchid PE]

I'm too busy to do that math. But if column B adds to 29, and column C adds to 21.9, then sure it's like that. Except they would probably round up C to 22.

Yeah they add up (Excel table).

That was what was confusing me. People said each question was weighted blah blah blah, then the exam said all questions are weighted the same. But it makes sense now that each question is weighted the same, and the weights are only used to determine the "difficulty" of the administered exam.

So the cut score meeting determines the weighting for an entire question bank, which is some number of problems much greater than 80? Then each administration pulls from that bank to created different tests?

That's what I'm wondering.

 

[RBHeadge PE]

the exam spec did not include NFPA 70E, NPFA 30B, NFPA 497, NFPA 499.

I'll add that they update the codes in the exam specifications all the time. Those updates don't precipitate a new cut score meeting.

 

[Orchid PE]

Summary of what I've learned:

• All questions are weighted the same when calculating your exam score.
  
• Each question is weighted differently when calculating the passing score.
  
• Professional Activities and Knowledge Study (PAKS) is used to develop the exam specification, and according to NCEES: “The results of this online survey will be sued to update specifications for the exam, which is used throughout the United States for licensing purposes.”
  
• There is a question bank where questions are pulled to produce exam sessions.
  
• The passing score weight of each question is is determined during an in-person committee meeting?
  

 

[RBHeadge PE]

 which is some number of problems much greater than 80?

The bank is certainly larger than 80 questions.

Then each administration pulls from that bank to created different tests?

yes

 

[Will.I.Am PE]

Well, I learned something, today.  arty-smiley-048:

 

[Tiger15]

I learned a lot. I'll probably forget it all because I'm currently sleep deprived.

 

[RBHeadge PE]

• Professional Activities and Knowledge Study (PAKS) is used to develop the exam specification, and according to NCEES: “The results of this online survey will be sued to update specifications for the exam, which is used throughout the United States for licensing purposes.”
  

And a survey design meeting precedes the PAKS meeting. The survey is sent to active PEs in that field.  The survey results informs the PAKS meeting.

• The passing score weight of each question is is determined during an in-person committee meeting?
  

Technically it's not a committee meeting. Non-committee members are required to participate in some number to avoid bias, group-think, etc.

 

[Tiger15]

The bank is certainly larger than 80 questions.

yes

I did not know this. I thought each exam was created, evaluated, and cut score determined individually. Creating one large bank based on the specification makes a lot of sense.

Another question @RBHeadge PE: Who creates the problems? I ask because Louisiana will give you CPD hours for "problem preparation for a NCEES or state professional engineering or land surveying exam." Seems to imply that it's something that "regular PEs" could do.

Edit: Or is this just put in to appease the NCEES gods and allow them to keep their licenses active? LOLOLOL

 

[LyceeFruit PE]

I'll add that they update the codes in the exam specifications all the time. Those updates don't precipitate a new cut score meeting.

They did plenty of other things to the Power exam spec for April 2018, the codes are the easiest thing to remember. i know Zach Stone has an article about all of the changes on his site somewhere but I'm lazy and others can google it if they wanna know all of the nitty gritty

 

[Orchid PE]

In case anyone comes across this in the future:

• All questions are weighted the same when calculating your exam score.
  
• Each question is weighted differently when calculating the passing score.
  
• After each pencil-and-paper exam administration, every exam undergoes extensive statistical analysis to determine the minimum level of performance required for entry into the profession.
  
• When the analyses and reviews are completed, NCEES changes the answer keys as necessary. The passing score and final correct answers for each exam are then used to score all answer sheets. A percentage of the answer sheets are manually verified, and the results are compared to the machine score to ensure accuracy.
  
• Professional Activities and Knowledge Study (PAKS) is used to develop the exam specification, and according to NCEES: “The results of this online survey will be sued to update specifications for the exam, which is used throughout the United States for licensing purposes.” A survey design meeting precedes the PAKS meeting. The survey is sent to active PEs in that field, and the survey results informs the PAKS meeting.
  
• There is a question bank where questions are pulled to produce exam sessions.
  
• The passing score weight of each question is determined during an in-person meeting (committee and non-committee members in attendance).
  

 

[LyceeFruit PE]

Wait how did you get the real cut score for October's power exam?!

really hope not lol

 

[Will.I.Am PE]

Oh the actual exam specifications.

 

[Tiger15]

NCEES should give tours

 

[RBHeadge PE]

Another question: Who creates the questions? I ask because Louisiana will give you CPD hours for "problem preparation for a NCEES or state professional engineering or land surveying exam."

Short answer, it's basically the PE committee of the respective engineering society.