Integrated Project Management (IMP) Training Course On

Integrated Project Management (IMP) Training Course On

Competitive Sealed Proposals AKA Best Value Procurement MDH Presenters Dana Dembrow MDH Chief of Procurement Attorney Former State Delegate 10 years as 1 of 3 Members of the Maryland State Board of Contract Appeals

Rendered formal decisions on Procurement Protests and Claims filed by vendors Authored almost all decisions (verdicts) during that time CSP - 4/2018 Joel Leberknight Retired State employee 38 years full time; 4 years part time. (17 years full time and 2 years part time at MDH) 35 years of Maryland procurement experience, including: Over 8 years as Chief of Procurement at MDH (DHMH) Almost 17 years as Chief of

Procurement at the Dept. of Budget & Management (DBM) Copyright 2018 Md. Dept. of Health - Unpublished Work 2 CSP DEFINITION COMAR 21.01.02.01.B.21 Competitive sealed proposals means the process set forth in COMAR 21.05.03 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 3 COMAR 21.05.03 Has only about 4 pages of substantive text on how to conduct these types of

procurements Much of this text is quoted in the Appendix But the practical aspects of CSP have been formulated over more than 30 years of protests and Md. State Board of Contract Appeals decisions And in this class are attempting to fully explain in over 500 slides CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 4 CSP History

When COMAR Title 21 first became effective on 7/1/1981 what is now CSP was called Competitive Negotiations Around 1988 the name Competitive Negotiations was changed to Competitive Sealed Proposals This was done because in Md. procurement we do not really negotiate As explained starting on slide 41 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 5 The Good & the Bad

This name change was good because it eliminated the confusion between what true negotiation is, versus what we do But, the name change was also bad because it removed the focus from the essence of competitive negotiation Now Competitive Sealed Proposals This CSP focus is described starting on slide 49 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 6 Historical Usage Until 10/1/2017, CSP was the preferred procurement method for Human, Social, Cultural and Educational Services and real property leases

See slides starting at slide 513 in the Appendix for descriptions of these types of services Prior to 10/1/2017, CSP could only be used for procurements other than for Human, Social, Cultural or Educational Services or real property leases if the Procurement Officer made a specific Determination CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 7 Determination

A determination is a written decision made by the Procurement Officer that is based upon written findings While the Procurement Officer makes the determination, the Agency Head or designee must concur with that determination Both the Procurement Officer and Agency Head or designee must sign the written determination to indicate their approval CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 8 Previously Required Determination Specifications cannot be prepared that would permit an award based solely on price And/or that

CSB is not practicable or is not advantageous to the State and there is compelling reason to use the CSP source selection methodology CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 9 Previously Required Determination Although it wasnt difficult to make such a determination for IT, many services and certain other types of procurement Frequently agencies would neglect to formally make the required determination And risk an audit exception by the Legislative

Auditors The determination could be 2nd guessed by the Legislative Auditors or a Control Agency in terms of whether the procurement at issue truly couldnt be done by CSB CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 10 When CSP Can Be Used Since 10/1/2017, a determination is no longer required to use CSP for anything other than human, social, cultural or educational services or real property leases Now the Procurement Officer just has to agree that CSP is the best method to use for a given procurement Although some formal evidence that the

Procurement Officer made that decision should be contained in the Procurement File i.e., a note to the file CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 11 When Is CSP Used CSP is frequently used for big ticket procurements Typically 40% of the annual dollar value of all State procurements are done via CSP More than is spent under any of the 12 other Title 21 procurement methods Almost all Information Technology procurements are

done by CSP Many regular services are procured by CSP CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 12 Flexibility (Advantages) of CSP Detailed specifications arent needed And are often discouraged It is only necessary to know the problem(s) to be solved, or need(s) to be satisfied Offerors are invited to propose a solution(s) to

the problem(s) or need(s) Offerors proposals can be very different in their approach to solving the problem(s) This is why price alone is not sufficient to determine which offeror should win the award Frequently, what the State is getting for each offerors price is as important as the absolute price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 13 Flexibility (Advantages) of CSP (Contd) Offerors can be allowed to correct flaws that would render them non-responsive under the Competitive Sealed Bidding method

This keeps offerors in the running for an award Within limits, specifications can be changed after proposals are received CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 14 Flexibility (Advantages) of CSP Oral presentations and discussions enable evaluators to get to know offerors and facilitate a thorough understanding of offerors proposals Through Best and Final Offers (BAFOs), offerors can revise their original proposal submissions Or, via multiple BAFOs, offerors can further revise already revised submissions

But written permission of the agency head or designee is needed for any BAFO after the 1st BAFOs can be requested for: Technical factors only The price only, or Both technical and financial factors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 15 More CSP Flexibility

Through discussions and BAFOs improved proposals should be sought from offerors that: i.e., the objective of discussions is to get an even better offer from all potentially qualified offerors than was in their original submission The award can be made to other than the lowest priced offeror Might be judged as being marginal Are also judged to have submitted good proposals To the offeror that is most advantageous to the State i.e., the offeror judged to have the best combination of product and price, or Best Value

The award is based upon subjective judgment CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 16 CSP Flexibility Example Legal research procurement by the Office of the Attorney General (OAG) in 2002 Written by Bill Kahn, Chief of Contract Litigation The only requirements were for offerors to: Identify the types of legal research they would provide

Provide a price for what was offered e.g., Per person accessing on-line information, per document, etc. State how long the prices were firm CSP - 4/2018 e.g., On-line, published works, etc. The duration of the contract was dependent on the period of time that offerors would hold their quoted prices Copyright 2018 Md. Dept. of Health - Unpublished Work 17 CSP Flexibility Example (Contd) The RFP stated neither alternate or multiple

proposals would be accepted Only 2-3 evaluation criteria were included If nothing is specified, there is nothing to provide an alternate to, or multiple of e.g., the duration of the contract Basically, the evaluation criteria distilled to, the OAG would pick whichever offeror was judged to give the best deal 4/2018 The technical offering was stated to be more important than price Mandatory & administrative clauses were included

Copyright 2018 Md. Department of Health (Unpublished Work) 18 CSP Flexibility Example (Contd) OAG received two proposals from the two primary vendors offering on-line legal research The proposals were formally evaluated by an evaluation committee There were written questions and requests for clarification, and a BAFO The firm selected for the award offered extensive services for rates that would be firm for 7 years 4/2018

i.e., it offered a 7 year contract Copyright 2018 Md. Department of Health (Unpublished Work) 19 CSP Flexibility Example (Contd) The OAG RFP: Contained mandatory clauses and administrative instructions Was only about 5 pages aside from the required clauses, with no specified requirements Had no pre-identified duration

Had no pre-prepared, structured price form Met the minimum requirements for a CSP Proposals were properly evaluated CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 20 CSP Flexibility Example (Contd) The OAG got a great deal Probably better than if the State had provided specified terms The OAG denied an award protest by the losing offeror

The protest denial was not appealed to the Maryland State Board of Contract Appeals (MSBCA) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 21 CSP Flexibility Example (Contd) This procurement was OK this way because: Only 2 national firms provided these services The scope of services was relatively narrow It was already well known what services these firms offered, and their typical pricing formats The only questions were:

Which of these services would an offeror bundle together For what duration would the pricing be firm CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 22 CSP Flexibility Example (Contd) This OAG legal research example is not the typical State CSP procurement But, it shows that under the right circumstances, functional or performance

specifications are better than design specifications And, it also shows the full flexibility of the CSP procurement method In its simplest form, all this procurement did was say, Make me an offer! 3/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 23 A 2 Edged Sword (It Cuts Both Ways) The above described flexibility is both an Opportunity and a Responsibility An Opportunity to obtain the Best Value A Responsibility to exert effort to seek or cultivate the Best Value Often the best value doesnt happen by chance It happens because State personnel set up the

conditions to get improved offers By eliciting information and improved proposals from offerors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 24 Contradiction State personnel can set up the conditions to get better proposals by aggressively using passive efforts This will be explained later We cant just tell offerors exactly what they

should include in proposals Other than what is in the RFP But we should continually tell them when they are not giving us satisfactory information, approaches, timeframes, staffing, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 25 Flexibility Gone Bad Agencies seem to fully understand that under CSP there is flexibility not to award a contract to the offeror with the lowest price In fact, at times agencies seem to exercise this flexibility to extremes

In some CSP procurements it seems that price is effectively rendered irrelevant This is taking a good thing too far Price should be an important award component, even under CSP Why this happens and how to prevent it in the future is discussed throughout this class CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 26 Sports Analogy 1 In any sport one team or participant might be dominant over another one for much

of the game or contest In such a situation the dominant team/participant wants to deliver a knock-out punch so that the opponent has virtually no chance of ultimately winning the contest CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 27 Sports Analogy 1 (Contd) Conversely the team/participant that is losing wants to keep it close i.e., avoid getting so far behind there is no realistic chance to achieve victory in the end As long as the score is close going into the

4th quarter, the later innings, the home stretch, etc. there is a chance for the trailing team/participant to get hot and grab victory from the jaws of defeat CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 28 CSP Parallel By using the processes described in this class you will be setting up the situation: Of keeping it close for offerors without the best initial proposal to improve their proposals so they might ultimately win But, also allowing even an offeror with a good initial proposal to land a knock-out punch by making its proposal even better CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 29 Sports Analogy 2 CSP can be equated to a marathon rather than a sprint It isnt who gets out of the starting gate the fastest Who submits the best initial proposal Its who crosses the distant finish line first Who ultimately provides the best value to the State

The best combination of technical proposal and price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 30 2 Out of 3 Isnt Enough There are 2 primary competitive procurement methods in Md. procurement Competitive Sealed Bidding, and Competitive Sealed Proposals Each has 3 words in its name Agencies understand the competitive & sealed words But there is not a clear understanding of the

3rd word The difference between bidding & proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 31 2 Out of 3 Isnt Enough (Contd) Until there is understanding of the differences inherent in these methods And how to derive the full benefit of the flexibilities listed on the preceding slides The benefits of CSP will not be maximized CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 32 Offer and Acceptance CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 33 Offer and Acceptance The concepts of Offer and Acceptance are key elements of General Business Law These are elements of Common Law and are formalized in the Uniform Commercial Code These concepts also are the foundation of most contracts CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 34 Offer In simple words, an offer is an expression of the willingness of one party (an Offeror) to do something specific of value for a 2nd party (an Offeree) in exchange for something of value from the 2nd party Both the offer and the something of value must be legal Contracts for illegal activities are not legally enforceable In business, the something of value from the 2 nd party is usually money CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 35 Offer The Offer must be unequivocal It must be certain both that an offer was made and what the offer entails Under Common Law, oral offers are enforceable if there is certainty and specificity of the offer Which often is not the case And has no bearing on State contracts which except for the most simple, low cost ones must be in writing The Offer can have conditions, such as: Be time limited Be to only 1 identified entity or be to one or more unidentified entities that satisfy the conditions

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 36 Acceptance If one party (an Offeror) makes a legally enforceable Offer to another party (or occasionally parties) The party to whom the offer is made (Offeree) has the right to accept or reject that offer Once there is unequivocal acceptance of an offer by the offeree, the offeror can be legally compelled to implement its offer

Unequivocal acceptance of an unequivocal offer produces a contract The acceptance has to be exactly what was offered CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 37 Lack of Acceptance Is Rejection If the party receiving an offer (the offeree) doesnt accept the offer: The proceeding can simply end; or, The offeree can make a counter-offer to the offeror

A counter-off means the original offeror is now the one that can accept or reject the counter-offer And, this process can continue back and forth until either there is acceptance of an offer from one or the other of the parties Or, there is rejection with no counter-offer This back and forth process is commonly called negotiating CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 38 Negotiations To negotiate means that one party makes an offer to another party

If that offer is unacceptable to the 2nd party, the 2nd party can make a counter-offer Or, sometimes multiple parties It tells the 1st party what it wants This offer, counter-offer process continues until there either is agreement or the negotiations fail CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 39 Negotiations (Contd)

In situations when ultimately there is agreement Often what is finally agreed to is very different from the positions either party started with Usually there is compromise by both parties Perhaps based upon a series of offers and counter-offers CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 40 We Dont Negotiate

But in government procurement we dont negotiate 1st, typically we are dealing with multiple vendors 2nd, aside from the specifications in a RFP we dont tell vendors exactly what we want & see if they will comply We dont make offers or counter-offers Because we dont negotiate the name was changed to avoid confusion (See slide 5) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 41 Solicited Vs. Unsolicited Offers

An offer can be either solicited or unsolicited If one party makes an offer to a specific 2nd party out of the blue, its an unsolicited offer Vendors sometimes make unsolicited offers to State agencies Which usually are not awardable unless there is a sole source situation Or, a party can invite or request other parties to submit offers to it This is a solicited offer And the document whereby the invitation is communicated is commonly called a solicitation CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 42 Solicitation When inviting offers, the inviting party can place conditions on how, when, etc. the offers can be submitted and what they must be for Per COMAR Title 21 (21.01.02.B (83), a solicitation: means invitation for bids, request for proposals, or any other method or instrument used to communicate to potential bidders or offerors a procurement agencys procurement needs In CSP we request proposals using a Request for Proposals (RFP) document CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 43 Procurement Document

Request for Proposals (RFP) RFP is commonly used as a generic term for any competitive procurement Technically, it should only be used for CSP procurements Invitation for Bids (IFB) is the correct document name for a Competitive Sealed Bidding (CSB) procurement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 44 Requesting Offers

When using CSP, State agencies invite or request vendors to make us an offer, in the form of a proposal We Request Proposals Hopefully, more than one proposal State requests for proposals are generally advertised and open to any qualifying vendor (offeror) to respond to Vendors determined to be sole sources for a given good or service can also be requested to submit a proposal, but this is not CSP CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

45 Accepting An Offer Under CSP We request or invite vendors to make us an offer (submit a proposal) We have the legal right to accept an offer (a proposal) and make an award if: The proposal conforms to the specifications The acceptance occurred within the period of irrevocability (120 days, etc.) Offerors cannot withdraw their offer during the period of irrevocability unless there is at least a 2nd BAFO request or we agree to the withdrawal CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 46 Accepting An Offer Under CSP Although not communicated as such, when we seek cures or BAFOs in offerors proposals, we are seeking revised offers We are requesting revisions to the original (or immediately prior) offer If offerors provide such revised offers, the new offer supersedes the prior offer See BAFO slides starting on page 267 CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 47 Accepting An Offer Conversely, vendors who submit proposals (make offers) can sue the State File a protest If they believe they satisfied the conditions of the request for proposals and had the most advantageous offer but the State did not follow through and award the contract to them i.e., They claim they had the most advantageous offer but we violated the terms of the RFP that says

we will award to the most advantageous offer The failure to award was because of bias or faulty evaluating CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 48 The Essence of Competitive Sealed Proposals GETTING THE BEST DEAL FOR THE STATE CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 49 The Essence of CSP The essence of CSP is to seek improvement in the proposals of possibly all offerors

Not just accepting what offerors, including good offerors, initially propose And, not immediately eliminating offerors with deficiencies Most agencies understand this concept from the price perspective They understand that usually offerors should be invited to submit a financial BAFO in an effort to get a lower price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 50 The Essence of CSP (Contd)

But many agencies & procurement personnel dont seem to understand this same concept applies to offerors technical proposals Obtaining improvement in offerors technical proposals benefits the State in two ways The obvious benefit (result) is there will be some increased level of contract performance The presumption is that even an offeror that submitted a good proposal can improve its proposal Even if only slightly And, sometimes more than slightly CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 51

The Essence of CSP (Contd) But perhaps a not obvious benefit is to keep more offerors in the running for the award Offerors whose proposals as originally submitted might have been determined technically unacceptable, potentially can revise them to make them acceptable Being judged to be technically acceptable means these offerors will remain in the competition and have their prices opened Which means more selection for the agency to choose among to arrive at the best value CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 52

Passive Negotiations We dont do this in an active way, such as in negotiations, by saying exactly how an offeror should change its proposal We should do it in a passive way, by saying what we dont like or dont understand And then allowing the offerors to try again to give us something well like better And often trying multiple times, improving the offer each time If at first you dont succeed, try, try again CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 53

Thats Not Fair! The reaction of many of you to the preceding slide is probably summed-up above: Its not fair to the offerors that submitted good proposals to begin with CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 54 Its Too Much Work Another typical reaction is that the process of telling vendors their weaknesses and allowing revisions, perhaps more than once: Is too much work

And, takes too much time to do CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 55 Why We Think This Way The reason for the 2 previously possible reactions Its unfair And, too much time and work Likely stems from people letting their understanding of Competitive Sealed Bidding (CSB) influence how they think Competitive Sealed Proposals (CSP) procurements should be conducted

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 56 Get the Best Deal for the State The objective of both CSB and CSP is to get the best deal for the State In CSB, the best deal, by definition, is the lowest bid That is responsive From a responsible bidder In CSP the best deal is the best value or Most Advantageous Offer CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 57 CSB In public procurement what we call CSB has existed much longer than what we call CSP So the notion of lowest bid wins is widely ingrained in peoples minds, including the public, vendors and State personnel not typically involved with procurement Many people also understand that the core principle of public procurement is Fair and Equal Treatment of vendors in competing for public awards CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

58 CSB (Contd) Since CSB existed long before CSP To a large degree the concept of Fair and Equal Treatment of vendors in competing for public awards has been framed in the context of what that means in CSB As stated earlier, in CSB the lowest bid wins But it is understood that awarding to the lowest bidder only makes sense if the bids are comparable Bids must be apples to apples, not apples to oranges CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

59 Sealed Bid Avoids an Auction Another core principle of CSB is that it is not an auction Vendors get 1 chance to bid That is why their bids are sealed and opened publicly Each vendors submits its best bid and hopes it wins There is no second chance No Second Bite of the Apple CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 60

Responsiveness The words Responsive and Nonresponsive came into being to indicate if bids are compliant with the requirements of the specifications, hence are comparable Are apples to apples Procurement people should know that: Responsiveness is determined solely on the basis of the submitted bid Without additional explanation or supplement i.e., as the bid came out of the sealed envelope Responsiveness doesnt apply to CSP

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 61 Summary of Key CSB Concepts Fair and Equal Treatment Responsiveness Judged solely on the basis of what is in the sealed bid envelope Only 1 chance to bid (1 bite at the apple) With rare exceptions you cant claim you made a mistake and even then you cant change your bid Especially to a price that is now the lowest bid Otherwise it isnt fair

Either because its an auction; or, There is a risk of the prior bids being leaked CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 62 A Little Knowledge Can be Bad Some of the highlights of CSB have been pointed out to illustrate how easy it is for persons who dont really understand CSP to incorrectly transfer what they know about CSB to CSP They incorrectly think offerors that dont get their initial proposals right should be judged non-responsive

Based solely upon their initial proposal As taken from the sealed envelope Otherwise an offeror has 2 bites at the apple Which is perceived as unfair CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 63 From the Past to the Present Now that weve talked about the past

About CSB How it came into being Some of its hallmarks And, about Competitive Negotiations and CSP Lets focus on the primary aim of this class Maximizing the benefits of CSP i.e., focusing on the difference between bidding and proposals The 3rd word in the title of the procurement methods CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 64

Question: How do You Maximize the Benefits of CSP? Answer: Change Your Thinking CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 65 Whose Thinking Needs to Change? Procurement Personnel Program Personnel involved in CSP

procurements Evaluation Committee members Agency Middle & Upper Management CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 66 How Should Thinking Change? People need to forget what they know about CSB when doing a CSP procurement Instead they need to learn and accept: The Objective of CSP

To get the best value That time and effort are needed to achieve that objective CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 67 Objective of CSP Get the best offer for the State (within reason) Getting the best reasonably possible offer is typically achieved by: Maximizing the number of offerors responding to the RFP Fully using the flexibility of the CSP process to: Avoid eliminating offerors for curable weaknesses or deficiencies

Get improved technical and financial offers CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 68 2 Assumptions 1. Every offeror can and will improve its proposal under the right circumstances Identification of weaknesses Maybe only slight ones for the best proposals Conscientious questioning Specific and generic With follow-up for answers that are unclear,

contradictory, incomplete or disliked Dont keep your thoughts to yourself If you think it, tactfully say it Issuance of cure letters Request for BAFOs CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 69 2 Assumptions (Contd) 2. The best proposal(s) will be unaffordable Dont just focus on that proposal to the exclusion of all others Use the process to help insure there will be other qualified offerors So that if the best proposal is unaffordable there is at least 1 viable alternative Not the best, but at least acceptable and affordable

And if the best offer is affordable you can be pleasantly surprised and accept it Plan for the worst and hope for the best CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 70 Less Isnt More Often agency personnel state that they want to save time and effort by only dealing with a few, clearly qualified offerors To achieve this mistaken objective of minimizing their work agencies:

Establish high minimum offeror response requirements to hold down the number of offerors Eliminate all but the best initial offerors as quickly as possible to get to the serious contenders Quickly open prices Then quickly make an award recommendation As explained in this class, this may not help achieve the objective of getting the best offer CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 71 Instead, More is Better

More is better in terms of: The initial competition The number of offerors Continuing competition More offerors being determined to be technically acceptable and having their price proposals opened The caliber of the competition More offerors ultimately judged to have good technical proposals Its how they finish that counts, not how they start CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 72

Why More Is Better Although saving time and effort seems like a laudable endeavor When it comes to CSP procurements frequently this premise is the equivalent of being penny wise and pound foolish While seeking to maximize competition may cost a few thousand dollars in person hours of work by involved State employees The potential payoff is hundreds of thousands, or millions of dollars of savings And/or better performance CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

73 Getting It Right The best way to illustrate that going through the effort pays off in the end is to cite real examples The 4 procurements cited on the following slides were done by MDH since 2005 3 of the 4 procurements involved the same procurement officer (P.O.) In 3 of the 4 procurements the P.O. awarded the contract to a different offeror than the evaluation committee recommended In each instance the agency head (the MDH Secretary) accepted the procurement officers decision

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 74 Getting It Right (Contd) All 4 decisions resulted in protests 3 of the protests were appealed to the Board of Contract Appeals The 4th contract award wasnt appealed In 2 of the appeals MDHs decisions were upheld In the 3rd situation the BPW approved the contract award notwithstanding protest and

the appellant withdrew its appeal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 75 Getting It Right (Contd) The evaluation committees recommendations were not accepted because the P.O. decided a different offeror represented the best value The P.O.s decisions were based upon a much greater price difference than the perceived technical difference In these 3 instances the P.O. went with offerors with much lower prices that were still technically very capable

Why the evaluation committees didnt reach the same conclusion is explored throughout this class CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 76 Getting It Right (Contd) Based upon the preceding litany of overruling evaluation committees, incurring protests, and fighting appeals at the Board of Public Works and Appeals Board you may wonder what was achieved for all this grief The answer: collectively over $35 million in lower prices Judged in the context of $35 million, all the

other expenses in terms of the time and effort of State personnel are trivial CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 77 Example 1 Clifton Gunderson vs. Myers & Stauffer from 2006 to perform auditing services for the Medical Assistance Program. Clifton Gunderson (CG) was the incumbent & was ranked #1 technically Myers & Stauffer (MS) was ranked 2 technically The rounded off prices were $26.6 million for CG and $17.3 million for MS

The exact price difference was $9,305,130, or 35% lower CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 78 Example 2 ACS vs. PSI from 2005 to provide Enrollment Broker Services for the Medical Assistance Program ACS was the incumbent and was ranked 1 technically, while PSI was 2nd technically The rounded off prices were $47.5 million for ACS and $41.9 million for PSI The exact price difference was

$5,670,139, or 12% lower CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 79 Example 3 IHAS vs. HCA to perform inpatient and outpatient audits of payments by the Medical Assistance Program to hospitals; IHAS was the incumbent and was ranked first technically, with HCA ranked 2. This contract started on 7/1/09 The pricing was a % of identified overpayments to be paid to the contractor IHAS quoted a 25% commission rate HCA quoted a 12% commission rate

It is believed this difference will equate to over $1 million less for HCA versus IHAS A 53% lower commission rate CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 80 Example 4 APS vs. ValueOptions to provide a Statewide Administrative Services Organization for publicly funded mental health services. This contract started 9/1/09

APS was the incumbent and was ranked 1 technically; Value Options was 2nd technically The rounded off prices were $71.1 million for APS and $51.6 million for Value Options The exact price difference was $19,443,425, or 27% lower CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 81 Using the CSP Process Fully and Conscientiously A Quick Resource Guide On How To Do It Maximize the Number of Offerors Responding to the RFP Do outreach to adequately advise potential offerors of the availability of the RFP Do focused, direct notice in addition to eMM

Relying exclusively on eMM may cause you to miss good vendors Minimize Minimum Offeror Requirements Perhaps only requirements of law or regulation Avoid excessive bond/insurance requirements Avoid overly restrictive specifications CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 83 Easier Said Than Done

Saying avoid excessive or overly restrictive requirements is easy Determining what is excessive or restrictive is much harder to do And, I know many persons believe not having minimum offeror requirements, bonds, etc., will allow unqualified vendor to submit proposals So, how do you learn what to do, or consider, or not do when drafting requirements? Many of the issues on the prior slide and many more are covered in the Writing Specifications procurement training classes CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 84 Do the 3 Step

(The Process, Not the Dance) Many years ago when I took a seminar on effective writing I was told to: 1. Tell the reader what I was going to tell them 2. 3. Then tell them the information Then tell them what I had told them Introduce the key points Again mentioning the key points The rationale for this approach was that

People learn by repetition It will help ensure that even if the details are missed or not comprehended the key aspects will be understood CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 85 The CSP 3 Step 1. Before the oral presentations (Orals) tell offerors the aspects of their proposal that are weak, confusing, or not justified 2. 3.

Preferably, tell them to reply to these aspects prior to the Orals If there is a time crunch, say these aspects will be discussed at the Orals Have the Orals and ensure that the previously noted information is addressed Allow offerors to provide additional followup after the Orals If despite Steps 1 & 2 any aspect of offerors proposals is still judged to be inadequate CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 86 The CSP 3 Step (Contd) By following this process usually offerors will provide additional information or responses Very few offerors will fail to make at least some attempt to address identified deficiencies

This is part of the process of eliciting additional information from offerors beyond what is in their original proposals If offerors dont respond, they have no excuse if they are judged not qualified or not the best value Even if offerors do respond, the responses still may be judged as not being adequate But at least the offerors had the opportunity CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 87 The CSP 3 Step (Contd) Hopefully, though, the responses will be judged adequate, thereby ensuring the continuation of offerors in the competition Sometimes, the additional information may even result in an offeror being judged to be a star performer, although the original submission didnt indicate that As described shortly, there are reasons why an offeror may not originally submit a good proposal, but subsequently can and will do so if

given the chance CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 88 The CSP 3 Step (Contd) Conversely not doing this may result in missed opportunities for improved offers, or lower prices or both Figuratively leaving money on the table And may actually prompt a protest when an offeror is told it wasnt selected because of an aspect that was easily curable by the offeror, or misunderstood by the State CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 89 Do You Do All This with All Offerors?

Ideally, without going through this process you should only eliminate offerors that: Are clearly incapable of satisfying the requirements of the RFP; and, Have virtually no possibility of becoming capable during the timeframe of the evaluation i.e., dont seek to eliminate as many offerors as possible, as quickly as possible CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 90 Procurement Conceit

Frequently we take 6, 8, 9 months or more to draft RFPs Then we give vendors 30 days or so to provide responses We expect the responses to be totally detailed and exactly as we direct Without regard for: Competing demands on the vendors They may also be responding to other RFPs The cost to offerors to respond CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 91 Offeror Response Costs

Frequently offerors expend thousands of dollars to respond to an RFP Sometimes tens of thousands of dollars Vendors need to expend their resources wisely in deciding which RFPs to respond to & how extensively to do so They may respond minimally if they: Dont feel they have a good chance at winning a particular award Recently learned of the RFP Are simultaneously responding to multiple RFPs

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 92 Have Some Understanding We should understand that there may be a big difference between the amount of effort a vendor will devote doing something on speculation Something they are not being paid for Responding to our RFPs Versus how they would perform on a contract they are paid to perform

So a minimal initial response may not mean that a vendor: Has little capability Would not fully, or even exceptionally perform a contract Would not quote a good price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 93 Have Some Understanding (Contd) The cost to vendors to respond to a RFP (CSP) is typically much higher than that of a bid (CSB) This is another major difference between CSB and CSP And why the rules and processes followed for

CSP are fundamentally different from CSB CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 94 Keep Offerors Engaged By going through the processes described in this presentation we tell offerors They are still in the running Exactly what we need more information about

And the more effort offerors put into proposals, even on an incremental basis The more they will continue to be involved and buckle down to win to justify their costs to-date And the more they continue in the running the better understanding we will have of their strengths and weaknesses CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 95 Get to Know Offerors The more you interact with offerors: The better you can assess their capabilities and commitment

Especially any you havent dealt with previously If you dont have this interaction the offeror may be just a name on a proposal You cant judge a book by its cover You likely will be reluctant to take a chance with an offeror you dont know well You probably will play it safe and go with a known (frequently the incumbent) offeror CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 96 Ignorance Can Be Costly Elimination of offerors for technical insufficiency without getting to know their capabilities has a cost We dont know how good they would have been

We dont know what their price was We pay for playing it safe Even if such offerors are not eliminated we still tend to not select them: We forego lower prices Sometimes substantially lower prices CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 97 Fully Use the CSP Process To use CSP to its full potential we must:

Use the pre-proposal conference as an active twoway communication tool Encourage questions and suggestions before, during & after the pre-proposal conference Without a rigid question cut-off date Check references Hold meaningful discussions with all offerors Invite a Best and Final Offer (BAFO) after discussions More than one BAFO can be invited CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 98

Hold Meaningful Discussions Allow adequate time for discussions Be prepared by thoroughly reading proposals Note areas of confusion, dislike, or if very good Earnestly prepare generic & specific questions Control the discussions Engage the offerors (get them to talk) Tell offerors when they are not answering adequately Ask appropriate follow-up questions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 99

Hold Meaningful Discussions (Contd) Tactfully let offerors know of any aspect of their proposals that are Deemed a weakness/not liked i.e., anything for which the offeror might be downgraded in the final evaluation Throw-out Gotchas (Got You) What I call Gotchas are generally minor flubs, usually buried in the proposal Just like DBM and DoIT have general RFP templates that agencies are to use and populate with particulars for their procurements CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 100 Gotchas! Often vendors have extensive generic proposal templates that they augment with customized information specific to a given RFP and liberal use of the name of the soliciting government Frequently, if you thoroughly read a proposal you can find a residual reference to another government or some statement that obviously doesnt belong E.g., We are pleased to offer this proposal to the Commonwealth of Virginia CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 101 Gotchas! If you, sort of humorously, throw out that you were mildly shocked to see on page X that this proposal was offered to, for instance, Virginia You are demonstrating that you thoroughly read the proposal to find such an obscure slip-up You might even ask if everything in the proposal was really meant for MDH, and not just left-over material for Virginia Doing this puts the offeror somewhat on the defensive And sometimes vendors will offer some minor extra to make up for the mistake CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 102 Gotchas! Ive even done this as a positive in really good proposals For instance, if I only find 1 typo, etc. in an entire proposal, I will congratulate the vendor and mention that on page X I found a typo and it was the only one in the entire proposal Again, I am evidencing a thorough reading of the proposal And positioning the offeror to want to maintain this positive perception with future cure responses, BAFOs, etc. CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 103 Gotchas Vs. Multiple Errors Its one thing to perhaps humorously point out 1 or a few mistakes in a proposal Its more serious if there are many spelling or grammatical errors, obvious missing words, references to other governments, etc. Especially if more than 1 government appears E.g., Virginia 6 places and Illinois 2 places In this type of situation the offeror should be

pointedly informed of such errors, including shown the worst ones CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 104 Gotchas Vs. Multiple Errors The Offeror should also be asked to explain why there are so many errors And perhaps seriously questioned whether all the content of the proposal is really in response to the Md. RFP, not one of the other entities mentioned CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 105

Hold Meaningful Discussions (Contd) Tactfully let offerors know of any aspect of their proposals that are Deemed a weakness/not liked i.e., anything for which the offeror might be downgraded in the final evaluation Allow the submission of written follow-up information and/or proposal revision via BAFOs This affords offerors the opportunity: To cure or improve upon deficiencies For an already adequate proposal to be made better

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 106 Hold Meaningful Discussions (Contd) The ultimate purpose of discussions is to improve the technical standing of all offerors, or as many offerors as is feasible The result should be for agencies to have more viable choices for the award recommendation Viable choices means more: Offerors under serious consideration for the award Opportunity for price to play a significant role in the award decision

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 107 Only the End Result Counts One offeror can: Submit the best proposal at the beginning: and, Seem to be the sure winner based upon its initial technical superiority And another offeror may have lots of omissions, ambiguities and weaknesses And have to make substantial changes to its technical proposal

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 108 Only the End Result Counts (Contd) There is no: Averaging or blending of the original proposal and the final proposal Consideration of how much one offeror had to improve versus any other offeror Regard for the amount of effort you had to invest to get to this point All that matters is that when the time comes to make an award selection, which offeror is now judged to be the best value

The most advantageous CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 109 Fair and Equal Treatment Fair and equal treatment in CSP means offerors were given the same opportunities The same opportunity to Cure deficient technical proposals Explain their proposals in oral presentations For technically qualified offerors to submit BAFOs CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

110 Fair and Equal Treatment (Contd) Fair and equal treatment also means if discussions, etc. are held with one offeror, they must be held with all offerors That have not been formally eliminated Told they were eliminated Fair and equal treatment doesnt mean How many proposal aspects need to be cured by one offeror versus another CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 111 Real-life Parallel Scenario You are shopping for a new car and the first dealer you visit quotes a very high price So you tell this dealer you will do some more looking And also might not have been very friendly You might even say there is no way you will purchase there But typically you will already have provided

your name, phone number, etc., just to get the price quote CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 112 Real-life Parallel Scenario (Contd) So you go to one or more other dealers in an effort to get the best deal A good price from a dealer you trust You may have eliminated the first dealer from consideration and be just about to inform another

dealer you will purchase there When the first dealer calls and quotes a much lower price than it did originally In fact, its price is now lower than any other dealers CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 113 Real-life Parallel Scenario (Contd) Also, the salesperson apologizes, apparently sincerely, for the rudeness or lack of attention from before Do you say? Sorry, its too late. You had your chance and blew it No. I owe it to dealer X to buy from them. They quoted me a good price from the start and treated me very well

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 114 Show me the Money! Or, do you do the equivalent of the Cuba Gooding Jr. character in the Jerry McGuire movie and say, Show me the money? i.e., it doesnt matter who quoted less originally Its who is going to charge less now For the same vehicle With other comparable factors

Immediate availability Cleaned and detailed Full tank of gas Delivered to your door All paper work properly done (Title, loan agreement) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 115 Show me the Money! If you go with the ultimate most advantageous offer in real life

Despite that offer not being the best originally Why wouldnt you do the same thing in your State job? But, even if you personally wouldnt buy from the dealer that ultimately offered you the best price, you are expected to go with the best value in a State CSP procurement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 116 Let Price Be Important If only 1 or a few offerors are judged to be technically competent only this/these offerors will be considered for the award Even if other offerors are judged to be reasonably susceptible of being selected for the award, without a cure process they wont

be under serious award consideration because of perceived technical shortcomings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 117 Let Price Be Important (Contd) This means that the agency will be almost forced into accepting whatever price this/these (few) offeror(s) propose If the price is reasonable, this is fine But if the price is not reasonable, the agency doesnt have a legitimate alternative The agency has to pay the too high price because it is not willing to select an offeror perceived as being of lower quality despite a much lower price CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 118 Let Price Be Important (Contd) But if through the discussion process other offerors improve their proposals so they are judged to be better than just marginal If the highest technically ranked offerors have prices that are judged unreasonably high The agency has a real alternative to select a lower technically ranked offeror whose lower price is determined to more than offset its lower technical capabilities CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 119

Let Price Be Important (Contd) And if the highest technically ranked offerors are only minimally higher in price The award can still be made to one of the highest technically ranked offerors Because overall it is judged to be the most advantageous CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 120 Disadvantages of Using CSP The Disadvantages of Using CSP Should be Considered Before Deciding

to Use CSP for a Given Procurement Disadvantages of CSP: TIME & EFFORT As previously mentioned, to properly do a CSP procurement takes lots of time & effort Often 3-6 months from the issuance of the RFP to a recommendation for award More time is usually allowed for offerors to submit their proposals than for bids Often 30-60 days for proposals vs. 21-30 for bids Proposals can be hundreds, or even thousands, of pages in length and be very complicated versus maybe a one page, or even 1 line, bid price

Every page of every potentially acceptable proposal needs to be read by every member of the evaluation committee, and usually by the procurement officer or contract officer CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 122 Disadvantages of CSP: TIME & EFFORT Very few, if any, CSP procurements are done on a dedicated basis i.e., almost always members of an evaluation committee and the procurement or contract officer still have other duties to perform They cant devote 100% of their time to reading proposals and the other required activities of CSP

Just as a chain breaks at its weakest link A CSP procurement progresses at the pace of its slowest member Usually the one with the most other duties, hence least availability, or is least affected by the outcome CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 123 Members of Evaluation Committees This is why there should be great care in selecting members of an evaluation committee that have both the time and

interest to do a proper job Dont select persons with titles that dont have time to read proposals or attend meetings Not the bosses The worker bees See the additional discussion of evaluation committees starting on slide 246 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 124 Disadvantages of CSP: TIME & EFFORT Submitting good proposals is costly for offerors The unexpected usually happens: e.g., Vendor questions take more time than expected to answer and distribute

Specifications amendments may be needed Sometimes decisions about changes need to come from higher management and there is difficulty getting their time and attention to get the decision Offerors usually ask for more time to prepare proposals than allotted, often weeks longer More vendors submit proposals than expected, and/or they are larger than expected CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 125 Disadvantages of CSP: TIME & EFFORT The unexpected usually happens: e.g., (Contd)

There may be a protest against the specifications Lots of cures or clarifications may be needed for offerors proposals Evaluation committee members get sick or have previously scheduled vacations, etc. Although not expected to be a problem under the original planned timeframes, delays mount-up, various planned dates slip, and the vacation time arrives before the award decision and causes further delay There can be weather related delays in the winter CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 126 Disadvantages of CSP: TIME & EFFORT

The unexpected usually happens: e.g., (Contd) There is difficulty scheduling orals Sometimes with offerors, but more often with evaluation committee members All evaluation committee members must attend all orals Lots of post-orals cures or clarifications may be needed Issues identified in proposals or at orals indicate a need for more specification amendments Which may mean offerors must revise their proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 127

Disadvantages of CSP: TIME & EFFORT The unexpected usually happens: e.g., (Contd) Members of the evaluation committee may have difficulty finalizing their technical rankings and/or overall rankings Prices are higher than expected, necessitating multiple BAFOs, possibly after revising specifications to delete some requirements The procurement officer or agency head may not agree with the evaluation committee recommendation or may require extensive revisions in the formal written recommendation for award memo (See Slide 322 for memo guidance) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 128 Disadvantages of CSP: TIME & EFFORT

The unexpected usually happens: e.g., (Contd) The offeror recommended for the award may have MBE, bonding, insurance, Good Standing, etc. issues or try to negotiate contract changes There may be difficulty scheduling debriefings The award recommendation may be protested CSP protests are typically more complicated than CSB The BPW may not be willing to approve the award recommendation without behind the scenes meetings or additional explanation CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 129

The Extreme Unexpected Happens After the presidential election of 2000, the 2001 session of the General Assembly passed a bill requiring a new voting system to be in place in Maryland by the 2002 gubernatorial election The bill was approved by the Governor around May 1, 2001, at which time it became law The primary election was to be in February 2002 In July 2001 DBM initiated the first in the country state-wide procurement for DRE (direct recoding electronic) voting machines (touch screen voting machines) for the State Board of Elections (SBE) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

130 The Extreme Unexpected Happens SBE claimed vendors were anxiously awaiting this procurement and could respond within 30 days But, upon issuance of the RFP vendors asked for 2 to 6 weeks more time to respond Due to the very short time to make an award and get machines by February 2002, only a 2 week extension was granted Proposals were due the end of the first week of September 2001 Working precincts were due a week later

DRE machines, printers, voter verification machines, etc. Everything needed to mimic a working voting precinct CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 131 The Extreme Unexpected Happens All offerors were located outside of Maryland and were sending their working precincts by air freight 9/11/2001 happened All flights were immediately grounded at the nearest airport and stayed there for about 3 days None of the offerors knew where there precincts

were or when they would get to Maryland They arrived about a week late (3rd week of September) The working precincts were evaluated by staff from local Boards of Elections with formal write-ups of their findings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 132 The Extreme Unexpected Happens There was also a 5 member Evaluation Committee

4 of the 5 members of the evaluation committee were from local Boards of Election or the IT Director for a given county The 5th member was from SBE Toward the end of October 2001, the 4 non-SBE committee members issued a joint letter saying the procurement should be cancelled and resigning Despite the law saying that the 4 counties with the most obsolete voting systems (P.G., Mont. Allegany & Dorchester 40% of the state population) had to have new ones by 2002, and the other 20 jurisdictions had to have the same system by either 2004 (17) or 2006 (3) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 133 The Extreme Unexpected Happens

In CSP the award is made by the Procurement Officer with the concurrence of the agency head Although virtually all CSP procurements have an evaluation committee, this is not an absolute requirement Since the formal precinct evaluation write-ups were available, along with information gained with the proposals, orals & cures, the procurement officer and 1 SBE evaluation committee member felt comfortable in continuing And the procurement continued with the opening of financials, a BAFO and an award recommendation CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 134 The Extreme Unexpected Happens

The law said that the State would pay 50% of the cost of the new system and the local jurisdictions had to pay the other 50% The cost for the four 2002 required jurisdictions was close to $17 million, or more than $8 million for the 4 counties Gov. Glendening would not proceed with an award until each of the 4 county councils & executives or board of commissioners agreed to pay their 50% Which happened the 2nd week of December 2001 And a hand-carried item was approved by the BPW a week later CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

135 The Extreme Unexpected Happens So the procurement that absolutely, positively, without fail had to be completed by the end of September 2001 and formally awarded in October 2001, was delayed about 2 months But the recommended vendor Diebold still got the needed machines and precincts delivered and working by the February 2002 primary election without serious problems Details about this Fast & Good procurement are in the Appendix, starting on slide 529 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 136

Why is CSP worth it? With All the Just Listed Disadvantages, Why Use CSP? Why Use CSP? Because the only alternative except for sole source or emergency situations is CSB Including the CSB variation of Multi-step Sealed Bidding (MSSB) Which means that: The agency has to specify everything it wants, often in great detail Sometimes we dont know exactly what we want

The lowest responsive bid from a responsible bidder has to receive the award There is no leeway for any other award consideration CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 138 Why Use CSP? It is difficult to detail in the specifications: A new, never previously procured requirement Exactly what should be in a contract if anything different is desired than what has always been done before Often vendors know more than we do about new trends, technology, processes, etc. With CSP they can communicate different approaches

in their proposals With CSB we have to tell them exactly what they must do, otherwise they dont have to do it and may not be able to price it CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 139 Why Not Use CSP But Take Shortcuts? I dont want to use CSB because I: Dont want to have to award only to the lowest bidder; or,

Cant write sufficiently detailed specifications to use CSB So why not use CSP but cut out steps and time? Why Not Take CSP Shortcuts? Why: Answer all the questions? Do amendments? Extend due dates? Do cures? Hold orals? Do BAFOs? Try at all to accommodate offerors availability for orals, debriefings, etc. Tell then when they have to do something and they do it then or else

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 141 Heres Why! (Not to Take CSP Shortcuts) Not taking the time and effort to get things right dramatically increases the risk of one or more of: Awarding to the wrong vendor Incurring a protest Incurring a contract claim after award Not getting desired performance under a contract because the contractor claims the specifications dont require anything else Paying more than we otherwise might have CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

142 CSP Cautions Picking an incumbent or other offeror without: Proper regard for price as an important award factor Using discussions to obtain better offers from all offerors that have a reasonable chance of properly performing what is being procured Can lead to an appearance of favoritism: Selecting the most advantageous offer is a subjective process It may appear that the inherent leeway in this subjective process was abused CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 143 CSP Cautions (Contd) Appearing to ignore price is potentially costly It may: Cause the State to pay more than it otherwise could have in the immediate procurement Cause a protest Cause offerors to cease responding in the future If they dont think they will win at any price Give leeway for an offeror that expects to win

regardless of competition to charge more CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 144 Selecting the Most Advantageous Offer Requires more than just: Reading technical proposals Then opening price proposals Then making an award decision Considerable effort needs to be exerted before getting to the award decision point Evaluators may not have enough information at this time to make a good award decision

You dont know what you dont know Pre-empts efforts to get better offers CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 145 HOW TO SELECT THE MOST ADVANTAGEOUS OFFER Use the CSP process fully and conscientiously Use the concept of worth (value)

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 146 Use the CSP Process Fully and Conscientiously (Contd) We tend to conclude that non-incumbents: Lack understanding of the problem to be solved Lack adequate detail in their proposals This may be true, or: It might just be a perception based on lack of

understanding of the offeror or its proposal Its like when you first tried to get a job (Catch 22) No one wants to hire you without experience But if no one hires you, you can never get experience We are more likely to eliminate an offeror for technical insufficiency that we dont know, than one that we do know CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 147 Use the CSP Process Fully and Conscientiously (Contd) Sometimes agencies seek to award contracts to

offerors with 100% to 300% higher prices than other, supposedly qualified, offerors: This usually means that: A proper determination was not made of the offerors capabilities; or, The lower priced offerors shouldnt have been determined to be qualified The failure to do a proper determination of offerors qualifications may cause problems later: In getting approvals by a control agency & the BPW By prompting a protest from a low priced offeror that you said was qualified but that did not receive an award

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 148 Use the CSP Process Fully and Conscientiously (Contd) On rare occasions it may be important to award a contract to the very best offeror Not just to a competent offeror, but the best one In such instances awarding to an offeror with a much higher price may be prudent But, this should be reflected:

In the relative technical and financial weights Technical weight should be much higher than the financial weight In the evaluation criteria CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 149 Use the CSP Process Fully and Conscientiously (Contd) Other times a large price differential is due to offerors proposing very different solutions to solve the States problem(s) Some approaches may be more intensive or extensive than others A large price variance may merely reflect this

differing level of effort Its OK to pay more if we are getting more CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 150 Use the CSP Process Fully and Conscientiously (Contd) Usually we: Want a quality vendor at a good price Are not specifically seeking to make the award to the: Best

offeror, technically Most expensive offeror Least expensive offeror If we want the lowest priced offer we should do a regular CSB procurement; or, Use Multi-step Sealed Bidding CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 151 Use the CSP Process Fully and Conscientiously (Contd) Instead we should make the award to the offeror with the best combination of product & price The best value The most advantageous offer

This could be the offeror that is: Highest technically ranked, or The one with the lowest price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 152 Use the CSP Process Fully and Conscientiously (Contd) But it just as likely could be neither of these: Not # 1 technically ranked offeror; or, Not the low priced offeror

Instead it is the offeror determined to be the best value (most advantageous) based on: What they will provide for the price they will charge, without having either of the # 1 attributes Not # 1 technically ranked offeror; or, - Not CSP 4/2018 the lowest Copyright 2018 Md. Dept. of Health - Unpublished Work priced offeror 153 Use the CSP Process Fully and Conscientiously (Contd) The key to the defensibility of any award

recommendation To a control agency To the BPW To unsuccessful offerors Is the demonstrated: Diligence in fully using the CSP process Analysis that went into making the award decision The recommended award must appear to make sense CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 154 Use the Concept of Worth

(Value) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 155 Personal Comparison Many of you probably obtained some type of higher education after graduating from high school You went to a community or 4 year college, etc. You probably did this to enhance your chances of getting a good job You (or your parents or others) paid money for your higher education In essence, you sought to buy into a better job by offering a future employer (the State)

skills & training for which this future employer didnt have to pay CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 156 Personal Comparison (Contd) To get your foot in the door you may even have: Taken an unpaid internship Done volunteer work in a certain field Taken a job for less than you wanted or thought you were worth By doing so, you low balled your salary CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

157 A Low Ball Price May Be Good Just as it may make good business sense for individuals to: Pay money to enhance their job marketability; or, Start for less than they think they are worth It frequently makes good business sense for an offeror to try to buy into a State contract: For expected long term future income

State contracts are frequently 3 years or more Incumbents are more likely to win future contracts To gain experience To gain a reference A State contract is frequently regarded as a valuable marketing asset (if its performed well) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 158 A Low Ball Price May Be Good (Contd) This doesnt mean that you should:

Always make an award to an offeror with a very low price You need to verify any offerors capability and reliability Or, Automatically disregard an offeror and not make an award just because of a very low price Do the proper diligence to see if an award is merited, even if a price seems to be too low CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 159 Conflicting Directions?

Some of the foregoing instructions may appear to be in conflict: On the one hand you are told: Dont eliminate offerors early Be inclusive, rather than exclusive Give them leeway to: Prove their capabilities Revise their proposals to make them better for the State More competition is better A very low priced offeror may be the best offeror

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 160 Conflicting Directions? (Contd) On the other hand you are told: Dont determine that clearly unqualified offerors are qualified: Eliminate them! Carefully check an offeror with a very low price to satisfy yourself that it can and will do the job for this low price

Just because they are the lowest priced offeror doesnt mean they should receive the award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 161 Conflicting Directions? No! It all comes down to: Fully using the flexibility of the CSP process Exercising due diligence in making conscientious, informed decisions It certainly takes considerable time and effort to do all this But the pay-off of being able to select the

best value offeror should be worth this effort CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 162 So How Do You Decide Which Is the Most Advantageous Offer? If points and weights are used, it is easy to identify the offeror that appears to be the most advantageous offeror The offeror wins that has the highest point total after factoring in the weights Weights such as 60% technical, 40% price, etc.

But the Office of the Attorney General (OAG) & DBM advise against using points & weights As does my experience As further discussed starting on slide 300 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 163 CSP Evaluation Guidelines How is the most advantageous offer determined when points & weights arent used? No hard & fast rules

If technical factors have more weight than financial ones, generally: Start with highest ranked technical vendor If this is also the lowest price vendor, go no further & make the award #1 technically ranked and lowest price is the ideal combination No other offeror can beat this combination CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 164 CSP Evaluation Guidelines (Contd) However, usually this # 1 & # 1 combination (highest technical rank and lowest priced offer) isnt what happens

If dont have this ideal situation: Start with highest ranked technical offeror Look at the price differential between this offeror and all other offerors Ask the question of whether this offeror is worth this price differential I.e., is this offeror worth the extra cost? Does this highest ranked technical offeror provide value for the extra cost? CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 165 CSP Evaluation Guidelines (Contd) If the answer is yes, go no further & make the award If the answer is no, keep going down the line in order of technical ranking until you arrive

at the offeror that: Is worth the extra price over any lower technically ranked offerors; Or, Is the lowest priced offeror You can award to the lowest priced offeror even if technical factors have more weight than financial ones CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 166 CSP Evaluation Guidelines (Contd) If financial factors have more weight than technical factors: Start with the lowest priced offer Determine if there is a compelling reason not to

recommend the award to this offeror If not, award to this lowest priced offeror If so, go to the next lowest priced offeror and ask the same question Continue until no higher priced offeror is worth its price differential over the preceding lower priced offeror CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 167 CSP Evaluation Guidelines (Contd) If financial and technical factors have equal weight, proceed under either of previously described basis Either: Work from highest technical ranked offeror

down; Or, Work from the lowest priced offeror up CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 168 Practical Ways to Get Better Proposals and Make Evaluation Easier HOW THE STRUCTURE OF THE RFP CAN ENHANCE EVALUATION CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 169 3 Way Synchronization: Mapping

Relying on the proposal evaluations and the discussion process to assure acceptable contractor performance Makes it more important to Properly inform offerors of all information to be included in technical proposals Construct evaluation criteria that are appropriate for effective proposal evaluation Have 3 way synchronization (mapping) between 1. The RFP objectives and deliverables 2.

The information offerors must provide in their technical proposals to display their capabilities and approaches 3. The evaluation criteria that will measure the offerors perceived ability to satisfy the RFP objectives and requirements CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 170 Mapping (Contd) For each objective or deliverable, by section specifically state what information is required to demonstrate capability or compliance

e.g., specified proposal information must relate to RFP section 2.2.1 Another piece of information will relate to section 2.2.2, etc. Also, for each evaluation criterion, specifically state which sections of the RFP are encompassed within that criterion Mapping is generally described, but not mentioned by name, in Section 5.4 of the Standard RFP CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 171 Prepare Specific Points/Questions

But mapping alone may not be sufficient Whereas section 5.4.5.6.2 of the MDH Standard RFP template instructs offerors to give a sectionby-section description of the proposed plan to meet the requirements of the RFP, i.e., a Work Plan Dont trust that all offerors will do this, or will do it to the level desired by the evaluation committee To help ensure better, more informative proposals and avoid the need for time-consuming cures, prepare specific points/questions to be addressed CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 172

Prepare Specific Points/Questions The RFP template has an optional section 5.4.5.6.6 that allows the addition of items offerors are to address I recommend that unless clearly unnecessary, wording such as the following be added at the end of 5.4.5.6.2: In providing the above required section-by-section description, ensure that all items included in section 5.4.5.6.6 are addressed, either as part of the 5.4.5.6.2 or 5.4.5.6.6 response, but not both. As appropriate, state whether the 5.4.5.6.6 items are addressed under the 5.4.5.6.2 response or the 5.4.5.6.6 one. Then word the section 5.4.5.6.6 items as desired CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 173 Prepare Specific Points/Questions In other words, determine exactly what information you want from offerors What questions you want answered What detailed descriptions you want What issues should be discussed Then put each question, issues, etc. in 5.4.5.6.6

Using either additional numbers or letters E.g. 5.4.5.6.6.1; 5.4.5.6.6.2; 5.4.5.6.6.3, etc.; or, 5.4.5.6.6. A; 5.4.5.6.6. B; 5.4.5.6.6. C, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 174 Prepare Specific Points/Questions Ideally, this step would not be necessary Ideally, in providing the section by section

descriptions requested by 5.4.5.6.2., all offerors would include exactly what we want But, the ideal rarely happens Offerors may think they are providing complete information, but issues we expected to see discussed or answered werent Or, were provided by one or a few, but not others E.g., an incumbent provides our desired information because it has gotten to know what we want, But non-incumbents dont have this insight, hence dont provide what we expect to see

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 175 Prepare Specific Points/Questions Frequently, the reaction of evaluation committee members to this scenario is to determine that the offerors lacking in detail: Have submitted incomplete technical proposals Dont understand the problem This is another version of the procurement conceit mentioned earlier

i.e., all offerors should understand exactly what we want, even if we dont say it Its also another subtle way in which incumbents may be favored Dont make offerors guess what we want; spell it out CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 176 Prepare Specific Points/Questions Even if we detail exactly what we want in proposals:

Various offerors will still respond differently Offerors with more knowledge or capability in the subject area, should still be able to demonstrate their greater knowledge and capability over other offerors And that is what is most important That better offerors can demonstrate they are better Not that some offerors figured out what we wanted to see while other offerors didnt Perhaps offerors that do have the knowledge and capability: Either didnt think to discuss one or more issues; or,

Were trying to comply with Standard Template clause 4.7, Economy of Preparation CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 177 Prepare Specific Points/Questions Std. Template clause 4.7, Economy of Preparation Proposals should be prepared simply and economically and provide a straightforward and concise description of the Offerors Proposal to meet the requirements of this RFP. Offerors have to guess how detailed and how

voluminous to make their proposals; i.e., Is less, more? Some offerors may believe that Evaluation Committee members become glassy eyed and resentful of overly thick proposals And that short and sweet proposals will be more favorably judged Or, is more, more? The more pages the better Throw but kitchen sinkWork CSP - 4/2018in everything

Copyright 2018 Md.the Dept. of Health - Unpublished 178 Prepare Specific Points/Questions By telling offerors the specific points/issues to be discussed, we are telling them not to skimp or fail to address the items in 5.4.5.6.6. i.e., they dont have to try to guess if less is more, or if more is more We are saying we want detail on these items Telling offerors what to include in their proposals

helps ensure that responses will be both more complete and more comparable, offeror to offeror Which means time & effort is saved that otherwise would be expended in constructing cure letters for all offerors & waiting a minimum of several days for responses CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 179 Prepare Specific Points/Questions Or, if cures are not requested: Offerors are evaluated on incomplete responses; or, Potentially qualified offerors may be rejected for an

incomplete submission Perhaps unnecessarily reducing the pool of potentially selectable offerors And may increase the risk of a protest from an offeror that was told it was eliminated or not selected because of a lack of detail that they easily could have provided if they would have known the information was wanted CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 180 Prepare Specific Points/Questions In federal procurements, agencies are required to tell offerors of any significant areas of their

proposal that are judged deficient and permit cures of those areas Agencies that dont do this automatically lose protests While federal procurement policy does not apply in Maryland procurements Telling offerors of significant weaknesses is still a good approach Attorneys for protesters always try to throw in federal procurement precedents in their protests And the Board of Contract Appeals may be swayed by such an argument and uphold a protest

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 181 Extreme Example of Incumbent Favoritism Many years ago, as I always try to do, I attended an evaluation committee meeting after orals, etc., focused on discussing which offeror should win I was shocked to hear the opinion of one of those in attendance who recommended awarding the contract to the incumbent because it gave us what we wanted, while the other offerors only gave us what we asked for in the specifications When I heard this I asked for an explanation

How could we not say in the specifications what we wanted? CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 182 Extreme Incumbent Favoritism I was told that the procuring unit had mistakenly included some requirements from the RFP two procurements ago, not what was in the RFP that resulted in the current contract Since the incumbent knew what it was doing currently and apparently had learned through its close working relationship with the agency that this was wanted under the new contract, it based its proposal upon the current contract requirements, not the current RFP specifications

Upon hearing this I said the RFP immediately had to be amended to change the specifications to what we wanted, with the allowance of revised proposals This was met with howls that there was no time for an amendment and revised proposals and re-evaluations The procurement had to be awarded promptly to avoid an extension CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 183 Extreme Incumbent Favoritism I said there was no time not to do this All a non-winning offeror had to do was learn it had been evaluated on non-existent specifications and it was guaranteed to have a winning protest

More importantly, it just wasnt right Which would mean either the same amendment would have to be done after the protest decision, or perhaps even a new procurement Which would take much more time than the expected 2 weeks for the amendment and new submissions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 184 Extreme Incumbent Favoritism

Despite such a dire forecast if an amendment were not done, the procuring unit appealed to the agency head to be allowed to proceed with the award with the current, wrong specifications The agency head knew better than to try to make an award under such flawed circumstances and directed the amendment be done and 2 weeks more time allowed for revised technical proposals After all this, the incumbent still won, but this time it was a legitimate selection Although the main issue in this situation was the perceived lack of time to do an amendment The incumbent would have benefitted from its knowledge and relationship with the procuring unit CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 185 Dont Specify: Evaluate

An effective way to reduce prescriptive or design specifications in a CSP is to make aspects that would have been specified into evaluation criteria i.e., instead of you telling the offerors what they must do Tell them what they will be evaluated on, and see how creative they can be E.g. let the offerors commit in their proposals to: The promised completion date The dollar value of Performance Guarantees Like the OAG Legal Research RFP from slide 17 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

186 Dont Specify: Evaluate If you change elements that normally would be specified into ones for offerors to say what they will do Add specific evaluation criteria that pertain to these elements Dont just bury the elements within existing criteria, such as the work plan E.g., have a criterion for time of completion and another one for performance guarantees Maybe make these evaluation criteria 4 & 5 After 1. work plan, 2. staffing & 3. offeror capability

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 187 RFP Structure Weighting of the Technical and Financial Proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 188 Basis For Award TECHNICAL PROPOSAL EVALUATION CRITERIA State the evaluation criteria State the relative importance of each criterion, using the Ranking Method Criteria may only be used to evaluate information

that the offeror has, or should have submitted Criteria should address all the information that the offeror has been told it must submit. Economic Benefit should be used as an evaluation factor when appropriate. CSP - 3/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 189 CSP Evaluation Guidelines In solicitations you must state the relative importance of technical evaluation factors versus price in making the overall award determination Award is to be made to the most advantageous offeror Most advantageous offeror (or Best

Value) is determined by combining technical factors and price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 190 CSP Evaluation Guidelines (Contd) Technical factors can be: More important than Price On rare occasions much more important Equally important with Price Less important than Price Price

can even be much more important than technical factors Since 2009, the most common scenario for services is that technical & price factors have equal weight in the award determination CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 191 CSP Evaluation Guidelines (Contd) Common Fallacy: The technical weight must be higher, or even much higher, than the financial weight to prevent

having to award to a poor quality offeror that low balls the price Reality: A poor quality offeror should not be judged to be responsible A poor proposal, especially after doing cures and orals, should not be determined to be reasonably susceptible of being selected for award This type of offeror should be eliminated from consideration and its price should never be opened CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 192 CSP Evaluation Guidelines (Contd)

If poor quality offerors/proposals are eliminated: It shouldnt be necessary to assist a high quality offeror by making technical factors much more important than financial factors So, even though the most common relative weighting scenario is for technical factors to be equal in weight with price When structuring the RFP award basis, consider whether it might be better for price to be more important than technical factors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 193 CSP Evaluation Guidelines (Contd) If the entire CSP procurement process is followed as described in this class:

Offerors judged to be unqualified, and/or Technical proposals ultimately (after cures, etc.) judged to be unacceptable Should be eliminated with their financial proposal never having been opened Offerors technical proposals usually are much closer in quality after cures, etc., than they were when originally submitted Which means price is more likely to be the deciding award factor Not absolutely, as in CSB But as a final factor among hopefully relatively technically close offerors CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 194 Previously Stated Caveats Dont seek to eliminate proposals or offerors rapidly May be better to let such offerors participate in discussions and technical BAFOs They may surprise you by demonstrating a better: Understanding of the problem to be solved Capability and/or approach to solving the problem Than indicated by their original proposal CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 195 Previously Stated Caveats (Contd) However, before opening financial proposals decide if the discussions, BAFOs, etc. are sufficient to allow you to select the offeror If not, dont open the financial proposal A low ball price may be good for the State If the offeror is capable of doing what is needed If you believe the offeror will actually perform what is needed for that price

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 196 Evaluating Proposals per COMAR 21.05.03.02 A & 21.05.03.03 A The evaluation factors and an indication of the relative importance of each evaluation factor, including price. The evaluation shall be based on the evaluation factors set forth in the request for proposals and developed from both the work statement and price. Technical proposals and price proposals shall be evaluated independently of each other.

(See Appendix 4 for more COMAR 21.05.03.02 A and 03 A requirements) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 197 Interpreting COMAR 21.05.03.02 A Evaluation factors and an indication of the relative importance of each, including price Has been interpreted to mean that the RFP only has to state: If the technical evaluation is more, less or equal to the financial evaluation (to the price)

Can use words such as, much more, substantially more, slightly more, etc. if appropriate to differentiate the technical and financial importance But the general default position is just that technical and financial factors have equal importance If a given technical evaluation factor is more, less or equal to other technical evaluation factors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 198 Interpreting COMAR 21.05.03.02 A For technical evaluation factors: Generally just state they are in descending

order of importance Could say: Are equally important One or two factors are much more important than one or two other factors But generally, should be relatively vague about relative importance Being vague allows more flexibility for the Department to select the most advantageous offer CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 199 Interpreting COMAR 21.05.03.03 A

Technical and financial proposals shall be evaluated independently This is why: Technical and financial proposals are separately sealed Technical evaluations are completed before financial proposals are opened Financial proposals are only opened for offerors determined to be reasonably susceptible of being selected for award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 200 Interpreting COMAR 21.05.03.03 A

This helps avoid an assertion that a given offerors technical evaluation and ranking was decided after its price was known so that a given offeror would win or lose i.e., that an offeror with a: High price would be technically ranked higher than it otherwise would have been to try to justify an award to it Very low price would be technically ranked lower than it otherwise would have been to try to justify not awarding to it CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 201 Separate Proposals

This is why offerors are warned not to include any financial information in the technical proposal And why the procurement officer should examine technical proposals to ensure no financial information has been included before giving them to an evaluation committee Sometimes financial prices: Are mentioned in the Executive Summary or elsewhere Or, can be calculated or guessed from information provided on MBE forms or economic benefit numbers CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 202 Inappropriate Price Information

If the procurement officer finds financial information in the technical proposal, he/she should remove the information before distributing proposals to the committee If it is obvious that something was deleted, only divulge to the members than inappropriately included financial information was deleted Or, an offeror can be instructed to submit revised pages that omit such information, with the revised pages being given to the committee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 203 Inappropriate Price Information

But even if some financial information is in a non-apparent place in the technical proposal and missed in the procurement officers preliminary inspection But then discovered by the evaluation committee in their thorough proposal review This does not cause the automatic elimination of the offending offeror The procurement is not tainted if one offerors prices are known but not other offerors prices And maybe not even if more than 1 offeror did this CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 204

RFP Structure Alternate & Multiple Proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 205 Alternate & Multiple Proposals Only applies to sealed proposals procurements In sealed proposals it must be stated whether either or both multiple and alternate proposals are permitted from offerors There can also be limitations on the portions of a proposal that are subject to multiple or alternate

approaches; i.e., 21.05.03.02 A (7); see Appendix 4. Only staffing levels, or Only staff experience levels, etc. If either or both are permitted it must be stated how these may/must be submitted CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 206 Multiple Proposals

Multiple Proposals means multiple ways to provide what is required by the solicitation The RFP can allow multiple proposals but limit the number that may be submitted; e.g., Only 1 extra proposal, or 2, etc. This may mean different levels of intensity with different associated prices; i.e., Perhaps a very intensive approach involving a lot of vendor resources and a corresponding high price Perhaps a moderately intensive approach with a moderate price Perhaps a minimal (bare bones) approach with a relatively low price

CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 207 Multiple Proposals (Contd) Offerors usually will not know how much the State has available to pay for a contract To try to avoid: Either pricing themselves out of an award; or. Submitting a proposal that is low priced, but that the State may judge to be too meager to effectively do what is needed Offerors may like the ability to offer multiple level-of-effort approaches

CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 208 Multiple Proposals (Contd) And, usually the State doesnt know how much it can get for the available funds Plus, frequently there is not an absolute cap on the available funds If the State knew that it could get increased performance for what might be regarded as relatively little additional cost, it might elect to find the needed extra funds

So, both offerors and the State may benefit if multiple proposals are permitted CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 209 Multiple Proposals (Contd) Or, different pricing levels may not be an issue Maybe there are different ways to satisfy the RFP requirements that arent very different in price But again the offerors dont know whether the State prefers one approach over the others

CSP - 4/2018 Offerors may prefer to show that they can perform the work however the State wants Copyright 2018 Md. Department of Health (Unpublished Work) 210 Multiple Proposals (Contd) But multiple proposals mean more work for both offerors and the State in terms of preparing and reviewing proposals Offerors usually are never required to submit multiple proposals But some may feel that the State expects multiple proposals if the State permits them

It may be better to not allow multiple proposals and have vendors determine what they think is the best single approach, and then defend that approach Issues of price, etc. can be worked out through discussions, BAFOs, etc. CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 211 Multiple Proposals If multiple proposals are permitted, usually the requirement is for offerors to identify one proposal as their Primary Proposal The Primary Proposal must be complete in all respects in

terms of responding to the solicitation Any other proposal would then be separately identified, such as Proposal 2, etc. Any proposal besides the primary one typically only contains information that is different from what is in the Primary Proposal (Contd) With the differences clearly highlighted All of these types of requirements, such as those mentioned above, must be stated in the solicitation CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 212

Alternate Proposals An Alternate Proposal means one that takes an exception to at least one requirement of the RFP By submitting an alternate proposal, an offeror is essentially saying that it thinks it can satisfy one or more objectives of the RFP better than by doing what the RFP requires The result of an alternate proposal supposedly will be a better result for the State in terms of: Price Product Or, both price and product CSP - 4/2018

Copyright 2018 Md. Department of Health (Unpublished Work) 213 Alternate Proposals (Contd) An alternate proposal may benefit offerors An alternate approach may allow an offeror to differentiate itself from its competitors, enhancing its chances of being selected for the award The alternate approach may be less expensive for the offeror, allowing it to quote a lower price i.e., an offeror may have more experience with a different approach, which means it can perform it better, for less cost But, to the State none of this matters

Unless what is good for offerors is also good for the State CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 214 Alternate Proposals As with multiple proposals, the RFP can limit the portions of the RFP that are subject to an alternative approach (which parts can have alternatives); e.g.,

(Contd) Staffing levels The location from which the contract will be performed A software system, or data connection method Or, the opposite tact can be taken, by identifying parts of the RFP that can not have alternatives There also can be a limit on the number of alternate proposals that can be submitted e.g., 1, or 2, etc. CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 215 Alternate Proposals (Contd)

If an alternate is allowed the RFP must state: Whether an offeror is permitted to only submit an alternate proposal Or, whether an offeror must submit a proposal that is fully compliant with the RFP i.e., only 1 proposal may be submitted 1 proposal And, may submit an alternative proposal(s), in addition to the fully compliant one A 2nd proposal, and possibly additional ones i.e., a minimum of 2 proposals: the primary one and the alternate(s)

CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 216 Alternate Proposals (Contd) The RFP must also state how the proposals are to be titled; e.g., Primary, or Fully Compliant proposal, vs. Alternate Proposal, or Alternate #2, etc. As with multiple proposals, the offeror should clearly state in its alternate proposal(s) What is different from the primary proposal Why it believes its alternate approach(s) is better for the State than what the RFP required

CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 217 Alternate Proposals (Contd) The allowance of alternate, and to a lesser degree, multiple proposals can lead to better outcomes than what the State initially specified But, with the greater flexibility afforded by alternate proposals comes the prospect of greater responsibility and effort Depending upon how radical an alternate

proposal is, it may be difficult for an agency to effectively compare it to the other proposals that are compliant with the RFP CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 218 Alternate Proposals Difficulty comparing perhaps very different alternate & regular proposals is a factor in why alternate proposals are rarely permitted But, this factor alone should not preclude the permissibility of alternate proposals 1st, as previously mentioned, there can be limitations placed on the aspects of the RFP subject to alternate approached

(Contd) Which might preclude radically different offers 2nd, an agency might have a default position that unless an alternate approach is clearly superior, it will not be considered for award CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 219 Alternate Proposals (Contd) Agency Default Position (Contd) If there are no limitations on the aspects of

the RFP subject to alternative approaches, the RFP might say that the agency reserves the right to only consider an alternate proposal that is clearly perceived to be a better solution for the State for the award i.e., if there is doubt about the viability of the alternate approach, the agency can notify the offeror that the alternate proposal is no longer under award consideration, without the need for discussions, etc. concerning that approach, even if discussions are held on regular offers CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 220 Alternate Proposals (Contd) Agency Default Position (Contd)

Such wording in the RFP permits the agency to decide if wants to hold discussions, etc. on an alternate proposal It can if it wants, but doesnt have to if it has no interest in the alternate approach Note that this option only pertains to alternate proposals versus regular proposals Among regular proposals, the usual rules apply that all must be treated the same, with the same opportunity for discussions, etc. CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 221 Alternate Proposals (Contd)

Agency Default Position (Contd) If the RFP doesnt have wording permitting different treatment of alternate proposals, they should be treated the same as any other proposal in terms of being the subject of discussions, cure letters, BAFOs, etc. i.e., the same amount of discussion and evaluation effort should be expended on an alternate proposal as for a regular proposal CSP - 4/2018 Which means more work for procuring personnel Copyright 2018 Md. Department of Health (Unpublished Work) 222 Alternate Proposals

(Contd) Agency Default Position (Contd) Assuming the RFP requires a base regular or conforming proposal, and not just an alternate one, when discussions are being held on the regular proposal, they should also be held on the alternate proposal This is particularly true for the oral presentation Typically, more time should be allotted for the oral presentation of an offeror with an alternate proposal that is still under consideration, than with an offeror with only a regular proposal CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 223 Alternate Proposals (Contd)

Contingency Planning If the initial decision is not to permit multiple, and/or alternate proposals, there should be discussion of the possibility of changing this decision after RFP issuance and prior to the award recommendation, based upon questions or suggestions from vendors, or other legitimate sources e.g., control agencies, etc., or circumstances arising within the agency since the RFP was issued that merit a reconsideration of the original decision CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work) 224 Multiple/Alternate Proposals (Contd) Conclusion

All of the issues discussed on the preceding slides in this section need to be considered when deciding whether to permit multiple and/or alternate proposals While it is true that multiple and alternate proposals can lead to more work for agency personnel and award complications These factors should not automatically result in a decision not to permit such proposals The State might benefit from the added variety of choice provided by such proposals CSP - 4/2018 Copyright 2018 Md. Department of Health (Unpublished Work)

225 The CSP Evaluation Process (A High Level Overview) And Guidelines Concerning Evaluation Committees INITIAL REVIEW (Performed by the Procurement Officer) The proper number of copies The technical and financial proposals have been submitted separately Minimum qualifications have been satisfied The technical proposal does not contain financial

information No obviously missing sections/information The proposal contains all required affidavits, MBE* & VSBE forms, etc. * an omission may be fatal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 227 Susceptibility When more than one proposal has been received, per 21.05.03.03 B, the Procurement Officer may initially classify the proposals as: Reasonably susceptible of being selected for award; or Not reasonably susceptible of being selected for award. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

228 Susceptibility Initially the Procurement Officer may take the default position of classifying any proposal that has the potential to be reasonably susceptible of being selected for award after cures and orals as being reasonably susceptible of being selected for award so that such offerors can participate in cures and orals Such a default position does not affect: A non-responsible determination about an offeror Making a not susceptible determination for a proposal judged to be so deficient that cures would largely constitute a totally new proposal CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 229 REASONABLY SUSCEPTIBLE OF BEING SELECTED FOR AWARD Restrictive Interpretation Only those proposals that have a good possibility of ultimately receiving the award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 230 REASONABLY SUSCEPTIBLE OF BEING SELECTED FOR AWARD Inclusive Interpretation All proposals that are at least minimally technically qualified, until the time when financial proposals are to be opened and considered CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 231 Susceptibility As detailed earlier, it generally is better to keep offerors in the running through cures and orals in order to better be able to judge their capability This is especially true if there are only 1 or 2 other proposals But in the final analysis an offeror should only be judged to be reasonably susceptible of being selected for award and have its financial proposal opened if it would be acceptable to be awarded the contract CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 232 Susceptibility In other words, if an offeror is determined to be reasonably susceptible of being selected for award and has its price opened And the price is much lower than other offerors such that it would be difficult not to award to that low-priced offeror But then the agency argues why the offeror should not receive the award, essentially because its proposal isnt really acceptable or the offeror isnt capable That should be the decision & prices not opened CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 233 Discussions After the initial meeting of the Evaluation Committee but before an award is recommended, if there are any questions or a desire to discuss the proposals with the offerors, make arrangements for all of the qualified offerors to meet separately with the Evaluation Committee. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 234 Discussions (Contd.) Although the RFP template includes a clause which allows the State to not hold discussions, in almost all circumstances it is recommended

that discussions be held. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 235 Discussions (Contd.) Per 21.05.03.03 C. (2), the purpose of holding discussions is to: Assure that the State has a full understanding of the offeror's proposal and ability to perform Assure that the offeror has a full understanding of the agency's requirements Obtain the best price for the State

Facilitate arrival at a contract that is most advantageous to the State. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 236 Discussions (Contd.) COMAR 21.05.03.03C(3)(a) states, Qualified offerors shall be accorded fair and equal treatment with respect to any opportunity for discussions, negotiations, and clarification of proposals. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 237 Discussions (Contd.)

Not required to ask each vendor the same questions. Provide all offerors with an adequate amount of time. Disclosure to a competing offeror of any information derived from a proposal of, or from discussions with, another offeror is prohibited. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 238 Discussions (Contd.) DOs You can say to an offeror that certain aspects of its proposal or exceptions taken to the State's requirements are

unacceptable and will result in the offeror's proposal being determined to be not reasonably susceptible of being selected for award. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 239 Discussions (Contd.) DOs Aside from notifying an offeror about "deal breakers," you can remind an offeror that its negotiation stance may have an adverse effect on its evaluation and rank CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 240 Discussions (Contd.) DOs

Always have the A.G.'s Office review the proposed contract language for form and legal sufficiency before finishing the evaluation of technical proposals. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 241 Discussions (Contd.) DONTs You may not negotiate the wording of a contract or any other substantive issue after the recommendation for award is made No Post-BAFO discussions You may not tell an offeror that if it agrees to certain changes, it will be recommended for award. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

242 Discussions (Contd.) DONTs You may not: Negotiate out clauses listed as mandatory in COMAR; but, you can change the wording of a clause when COMAR states that the wording is suggested, but not mandatory CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 243 High Level Evaluation The purpose of an Evaluation Committee is to advise the Procurement Officer and the Agency Head about which proposal(s) represents the most advantageous offer(s) for the State. (See slide 530)

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 244 High Level Evaluation (Contd.) Select only Evaluation Committee members who will dedicate the necessary time required to do a proper evaluation, including a commitment to read all proposals and other materials and attend all of the evaluation proceedings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 245 High Level Evaluation (Contd.) The Evaluation Committee must evaluate technical proposals independently of financial proposals. Each member must read each proposal.

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 246 High Level Evaluation (Contd.) Confirm that each evaluator has a copy of the RFP and any materials sent to prospective offerors, such as addenda. Each evaluator must read all those documents before reading the proposals. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 247 High Level Evaluation (Contd.) Distribute at least one copy of the Evaluation

Sheets. Review the instructions with the committee Develop a schedule of the evaluation work to be done CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 248 High Level Evaluation (Contd) Each evaluator should write the rationale for his or her evaluation of each proposal on the Evaluation Sheets, including both positive and negative aspects, and discuss their evaluation with the other members. Since any such notes or rationale might ultimately be provided to the attorney for a

protester, they should not contain any potentially inflammatory wording or indication of positive or negative bias toward any offeror CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 249 High Level Evaluation (Contd) When all discussions with qualified offerors about the nonfinancial aspects of their proposals, including the scope of work, are finished and all BAFOs received, the committee should formally assign a technical ranking for each proposal (The committee needs to make a final collective decision about the ranking of the various Technical proposals) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 250

High Level Evaluation (Contd) After separately ranking the Technical and Financial proposals, the Committee should determine which overall proposal (combined technical and financial) is most advantageous to the State and the relative positions of the other overall proposals. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 251 Evaluations per COMAR 21.05.03.03 A 4. 5. 6. Numerical rating systems may be used but are not required. Factors not specified in the request for proposals may not be considered.

Initial evaluations may be conducted and recommendation for award made by an evaluation committee. Final evaluations, including evaluation of the recommendation of the evaluation committee, if any, shall be performed by the procurement officer and the agency head or designee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 252 Tied Proposals (Not) It is virtually impossible for there to be tied proposals While occasionally there might be technical proposals that are essentially tied

Judged to be the same I know of no instance when there have been tied financial proposals in addition to tied (equal) technical proposals Even if there are only slight technical or financial differences, any difference means there isnt a tie CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 253 Evaluation Committees CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 254

Evaluation Committee Although per point # 6 on slide 252, there is no absolute requirement for an evaluation committee to be used in CSP procurements, I have never undertaken a CSP procurement without one The Procurement Officer: Is supposed to know Maryland procurement procedures Typically is not an expert in the subject of various procurements Does not have to have a close working relationship with the selected offeror An evaluation committee is usually comprised of experts from the customer agency, perhaps with

some stakeholder representation CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 255 Evaluation Committee (Contd.) An Evaluation Committee should consist of at least three persons Having more than five persons on the Evaluation Committee often creates difficulties with scheduling Usually committee members are employees of the procuring agency, but this is not a requirement; committee members can be from: Another State agency Another level of government The private sector

e.g., a stakeholder CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 256 Evaluation Committee (Contd.) Key considerations for deciding on nonState employee committee members: The time availability of non-State personnel State employees, even ones from other than the procuring agency, can be directed to serve on the committee as part of their job This is not the case with non-State personnel They should agree to evaluate based upon the best interest of the State Even if what is best for the State may not be in the best interest of the group, or entity they

represent CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 257 Evaluation Committee (Contd.) Non-State committee members certainly can express the view of what would benefit their constituency, with the prospect that common ground can be found and the best vendor for the State will also be the best for the stakeholder But if this isnt the case, the purpose of the procurement is to obtain the best deal for the State, not non-State interests It might be difficult to find stakeholder personnel that will agree to this premise

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 258 Evaluation Committee (Contd.) Although it is rarely the case, there is no prohibition against paying travel or other expenses, or even an hourly rate, for nonState personnel serving on a committee The State personnel and consultants get paid for their time spent and their expenses, so it is only fair to do so with non-State personnel that require this as a condition of serving on the committee As long as they agree to the other conditions of no conflict of interest, time availability, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

259 Evaluation Committee (Contd.) There can be consultants or technical advisors to the evaluation committee and procurement officer These would not be voting members of the committee, but would have some particular expertise that would augment the skill set of the committee members for complex procurements Such technical advisors usually are paid on a labor hours basis for all effort exerted on behalf of the committee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 260

Evaluation Committee (Contd.) Aside from providing technical expertise, such advisors might also perform specific activities concerning the procurement to save time and effort for the State employees This might help speed the evaluation when State personnel cannot be exclusively dedicated to the evaluation, or when resources are otherwise limited; e.g.: Performing reference checks Keeping minutes of meetings and proceedings Drafting cure letters & questions for offerors Analyzing financial statements Analyzing complex financial proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

261 Evaluation Committee (Contd.) For most agencies, the Procurement Officer (P.O.) should attend and participate in all Evaluation Committee proceedings, including committee meetings and offeror discussions The P.O. should not be an official, voting member of the committee The P.O.: Receives the recommendation of the evaluation committee; and Makes the award recommendation to the agency head or designee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 262 Evaluation Committee (Contd.)

It doesnt make sense for the P.O. to on the one hand have the authority to accept or not accept the committee award recommendation And, on the other hand only be one of the evaluators that makes the recommendation, ostensibly with no more say than any other member In other words, the P.O. should act more like the way in which a judge presides over a trial CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 263 Evaluation Committee (Contd.) i.e., advises the committee what it can do, or

consider and not do, or not consider Much like a judge instructs a jury of what can be considered: the applicable law; what testimony or evidence can be considered in the verdict versus, stricken from the record And ultimately decides if he/she can accept the committees recommendation Much like a judge in certain circumstances can vacate the decision of a jury Or, decides to cancel and redo a procurement Like a judge can declare a mistrial & new trial CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 264 Evaluation Committee (Contd.) MDH is unique among agencies in that a single person Dana Dembrow is the P.O. for virtually all non-small procurements In MDH the Contract Officer usually substitutes for the Procurement Officer in terms of reading proposals and attending evaluation meetings, orals, etc. But does not substitute in making formal determinations, recommendations & decisions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

265 Evaluation Committee (Contd) At the end of the evaluation process, each member of the evaluation committee should sign the final write-up of the evaluation to indicate this was the recommendation of the committee Even if the recommendation was not unanimous, that fact should be noted in the write-up and each should sign indicating agreement with the write-up The Legislative Auditors look for this signed document if they review a given CSP procurement, even though the evaluation committee does not make the award And will note an audit exception if this document is not in the procurement fie CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 266 Best And Final Offers (BAFOs) What To Do and Not Do CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 267 BAFO Applicability Only allowed in Competitive Sealed Proposals (CSP) procurements Must involve all offerors not formally eliminated

Generally: Not required for any given CSP procurement But, a good idea for virtually any CSP procurement Involves only 1 BAFO request Can have additional BAFOs with written determination by the agency head or designee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 268 What Does It Do? A BAFO changes an offerors proposal in some fashion

The BAFO submission replaces (supersedes) the original or prior submission Offerors are not permitted to revise their proposals at will They may only do so upon the invitation of the procurement officer CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 269 A BAFO Changes the Proposal

During discussions offerors should be informed of major deficiencies and permitted to try to cure them Or, an offeror may simply want to sweeten its deal (offer) A BAFO allows this to happen Ensures original financial offer prices can be changed to be consistent with any revisions to the technical offer subsequent to the original technical offer submission CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 270 Change in the Proposal Generally is a positive in that it permits deficiencies to be cured Can be a negative in that on rare occasions an initially acceptable proposal can be rendered unacceptable with a BAFO

If an offeror takes new exceptions, introduces a conflict of interest, submits non-compliant BAFO MBE forms, etc. It can be eliminated; Or, Except for MBE deficiencies, informed of this new deficiency & another BAFO requested CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 271 BAFOs & MBE Impact Usually a BAFO should have no impact on MBE participation The participation should be as examined with the original submission

But if there is a significant change in the scope of the RFP that was initiated via an RFP amendment It may be advisable to allow offerors to reconfirm or re-submit the MBE forms CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 272 Generally an Invitation Usually offerors can either Provide a BAFO; or, Decline to do so The absence of any response from an offeror is the same as declining to submit a BAFO

A response can also be that an offeror wants to withdraw from the procurement If more than one BAFO has been requested Sometimes even with a 1st BAFO request CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 273 Mandatory vs. Optional Generally BAFO submission is solely at the offerors discretion However, if there has been a substantive amendment to the RFP issued after proposals were received, a BAFO can be required Ensures the proposal is now based upon the changed circumstances An offerors failure to respond can be

construed as withdrawing from the procurement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 274 Submission Dynamics Establish a set time, date & place for a BAFO submission But, with the possible exception of MBE requirements, there is no rigid submission deadline requirement Can accept a BAFO after the established time

Can reject a late BAFO but then immediately request another BAFO Submission can be electronic (email) if the BAFO instructions so state Should be followed up by a hard copy submission CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 275 BAFO Scope Can be restricted as to what can be changed Generally involves the financial offer only; but, Can be technical only Can be both technical & financial Can be limited to just a single facet

Penn Parking Board of Contract Appeals case BAFO was limited to the correction of an error in the RFP & only offerors who detrimentally relied on the error could participate in the BAFO, and only to the extent of correcting amounts caused by that error CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 276 Not a Cure-all A BAFO cannot be used to allow offerors to cure something that is not curable e.g., if an offeror did not properly submit MBE forms with its initial proposal and the proposal was deemed not reasonably susceptible of being selected for award, or the original proposal was submitted late The offeror must be eliminated Once it is eliminated, it is gone

It cannot come back to life via a BAFO CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 277 Check with AAG It is a good idea to have an involved AAG review BAFO issues, including: Proposed BAFO invitation wording, Proposed action regarding a particular offerors BAFO, Who can participate in a BAFO (pre-BAFO rejection), Who is eliminated as a result of a BAFO CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

278 Real Examples of Award Decisions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 279 Real Examples Insurance for employees, retirees & dependents Personal Accident & Dismemberment (PA & D) Group Term Life (Life) Technical factors were worth more than

financial factors in award determination CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 280 Real Examples (Contd) Metropolitan Life Insurance Company was: Lowest priced offeror in both procurements Relatively low ranked technically for both Despite this relatively low technical ranking, Met Life was still judged to be a very good company

5th out of 6 for PA & D 6th out of 8 for Life Met Life had had the Life contract for 11 years All offerors were solid, reputable companies Received award for PA & D Didnt receive award for Life CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 281 PA & D Metropolitan Life Insurance Company was: From 26% to 33% lower in price than the 4 offerors that were ranked higher technically

$1,851,699 to $2,554,027 lower in price $2,045,862 lower than # 1 technically ranked firm # 1 technically ranked firm had 4th lowest price 28% higher price Awarded the contract to Met Life None of the 4 higher technically ranked offerors were judged to be worth 26 to 33% higher price CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

282 Personal Accident & Dismemberment Insurance (9/4/02 BPW) Vendor Technical Rank $ Cost Financial Rank Overall Rank Met Life 5 5,290,567 1

1 Std Ins 1 7,336,429 4 2 MN Life 2 7,766,539 5 3 Prudential

4 7,142,266 3 4 Amer Intl Group (AIG) 3 7,844,594 6 5 CNA Fin 6 6,485,508

2 6 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 283 LIFE Metropolitan Life Insurance Company was: From 4.2% to 15% lower in price than the 5 offerors that were ranked higher technically $1,259,543 to $4,959,198 lower in price $1,259,543 lower than # 1 technically ranked firm # 1 technically ranked firm had

Not awarded the contract 2nd lowest price 4.2% higher price #1 technically ranked offeror was judged to be worth 4.2% higher price Also, note that offerors 1 & 2 were tied technically, but not in price or overall CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 284 Group Term Life Insurance Vendor

Std Ins MN Life CIGNA Met Life Prudent Amer In CNA UNUM CSP - 4/2018 (10/2/02 BPW) Tech $ Cost Cost Rank Rank 1 1 3 6 5 4 7 8

29,724,890 33,083,742 32,670,383 28,456,347 32,426,766 33,424,545 31,184,581 35,179,627 2 6 5 1 4 7 3 8 Copyright 2018 Md. Dept. of Health - Unpublished Work Overall 1 2 3 4

5 6 7 8 285 A Real Most Advantageous Offer Real Procurement Example # 3 In the 2 previous examples the award was made to: # 1 financially ranked (lowest cost) offeror (PA & D) # 1 technically ranked offeror (Life) In next example, out of 10 offers the award was not made to either # 1 ranked offeror Not to # 1 financially (lowest price) Not to # 1 technically

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 286 Real Procurement Example # 3 Vendor Tech Rank Cost Rank $ Price Overall A 1 8 3,477,700 3 B

2 9 3,714,693 5 C 3 2 1,397,953 1 D 4 3 1,707,079

2 E 5 5 2,448,141 4 F 6 10 3,829,015 10 G

7 4 2,007,863 6 H 8 7 3,150,360 8 I 9 6 2,966,265

9 J 10 1 966,396 7 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 287 Real Procurement Example # 3 (Contd) The selected offeror was: Technically # 3 (technical worth more than financial)

2nd lowest price RFP would result in Labor Hours contract Contractor gets paid for actual hours worked Impossible to estimate in advance # of hours needed RFP had a price form with a model of assumed quantities that all offerors had to use Price difference was due to offerors quoting different hourly rates for specified labor categories Only variable was the hourly labor rates CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 288 Real Procurement Example # 3 (Contd) #1 & 2 technically ranked offerors had very high labor rates, thus their model price was very high Average hourly wage rate was more than twice the average per hour cost of # 3 ranked offeror The staff being offered by #1 & #2 werent judged to be twice as productive as the personnel of the # 3 technically ranked offeror Neither the #1, nor the #2 technically ranked offerors were deemed to be:

Worth the high hourly rates The most advantageous offeror CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 289 Real Procurement Example #3 (Contd) After #1 & 2 technically ranked offerors were: Judged not to be worth their high prices; hence, Determined not to be the most advantageous offeror The # 3 technically ranked offeror: Was higher ranked technically than the 7 other offerors Was lower in price than 6 of the 7 remaining

offerors Since it was the 2nd lowest priced offeror Only had to be compared to the lowest priced offeror, which was # 10 technically CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 290 Real Procurement Example #3 (Contd) Lowest priced offeror was #10 technically About 30% hourly rate differential between #3 technically ranked offeror & #10 offeror

#3 was deemed to be worth this differential over #10 (lowest) technically ranked offeror Overall work plan & organization resources were better Personnel were judged to be more experienced & knowledgeable Presumed to be more productive Would use fewer hours & produce better result #3 technically ranked offeror was selected for award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 291 CSP Myths Vs. Reality & Issues and Answers

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 292 Issue Must you always check references even if you have direct experience with the offeror and know it well? Answer You still should check outside references even if you are a reference. Although a vendors performance with you is more important than what it has done for others, It is still advisable to consider that outside references could provide information that indicates that the offeror is beginning to have problems that might relate to the offeror's capability to perform your contract or call into question its integrity or reliability. It is also more defensible if there is a protest to have additional references to support your conclusion, good or bad CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

293 MYTH You must check all the references that an offeror gives you. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 294 MYTH You must check all the references that an offeror gives you. REALITY You only need to check as many references as you think are necessary to make a determination; however, when the references are used as an evaluation factor you should attempt to obtain a reference from the same number of sources for each offeror. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 295

MYTH You may only use information obtained from the same number of sources for each offeror. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 296 MYTH You may only use information obtained from the same number of sources for each offeror. REALITY You should try to obtain references from the same number of sources; however, if some of those sources for one offeror cannot be contacted, do not respond or decline to provide a reference, yet all of the sources for another offeror do respond, you can still use the information from all the references. Similarly, if one or more offerors provided fewer references than other offerors, you are not restricted to contacting the fewest number of references provided. CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 297 MYTH You are limited to checking only the references that the offeror has included in its proposal. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 298 MYTH You are limited to checking only the references that the offeror has included in its proposal. REALITY You can use any knowledgeable source, regardless of whether the offeror included that source. CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 299 References Whether information obtained from references must be publicly disclosed or disclosed to the offeror to whom it pertains is a Public Information Act issue Consult with the Office of the Attorney General for assistance with such a request Allow an offeror an opportunity to refute a negative reference if the information from the reference may impact the offeror's ranking or the responsibility determination

If possible, tell the offeror about the negative reference in cure letters or at orals, perhaps anonymously if the reference has so requested, after checking AAG for OK If at orals, allow a follow-up response CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 300 MYTH If a member of the Evaluation Committee is named as a reference, that person must resign or be removed from the Evaluation Committee. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 301 MYTH If a member of the Evaluation Committee is named as a reference, that person must resign or be removed from the Evaluation Committee.

REALITY There is no inherent problem with an Evaluation Committee member being named as a reference. An evaluator named as a reference does not mean that the evaluator cannot render a proper evaluation. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 302 Myth Vs. Reality Ranking Vs. Qualitative Labels MYTH: When evaluating proposals it is necessary to use qualitative (adjectival or descriptive) labels Excellent, Very Good, Good, Fair & Poor - for each offeror for each evaluation criteria and for the overall

ranking REALITY: Each of us have our own opinion of what constitutes Excellent versus Very Good, etc. And there are gradations within each of these labels CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 303 Myth Vs. Reality Ranking Vs. Qualitative Labels How much qualitative distance is there between an offeror that just barely is rated excellent versus one that is solidly very good? i.e., in school, 90-100 might be an A which equates

to Excellent 80-89 might be a B which equates to Very Good With just Excellent versus Very Good it isnt known whether the Excellent scored a 100 versus a 90. And it isnt known whether the Very Good was an 89 versus an 80. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 304 Myth Vs. Reality Ranking Vs. Qualitative Labels Even with these labels there might be a very substantive spread of 100 versus 80 Or, a non-substantive spread of 90 versus 89

With just these labels it isnt known how much better an Excellent was versus a Very Good There still needs to be some description that both explains why a particular offeror received one rating versus another, and how much better it is e.g., Offeror A was barely judged to be Excellent and Offeror B was solidly judged Very Good, and almost rated Excellent CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 305 Myth Vs. Reality Ranking Vs. Qualitative Labels Versus, Offeror A was judged to be Excellent in all criteria and was far superior to all other Offerors,

including Offeror B, which barely rated a Very Good And qualitative labels can serve as targets of complaint by offerors that regard themselves as better than their label might indicate i.e., an offeror rated as Very Good may think it should be Excellent; or, Offerors rated Fair may think they should be Very Good But an offeror ranked 2 or 3 or 4 has no idea how close or how far it was from the offerors ranked above it All it knows is 1 or 2 or 3 offerors were judged better CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 306 Myth Vs. Reality

Ranking Vs. Qualitative Labels Obviously, a much greater differentiation is evident between Excellent and Good (70-79) But there still needs to be a narrative explanation that indicates the actual magnitude of spread i.e. is it 100 versus 70 or 90 versus 79? Deciding to pay much more money for an offeror with a 100 versus a much lower priced offeror at 70 is much more defensible than the same recommendation of a 90 offeror versus a 79 one And presumably an offeror with an overall rating of Poor would not be judged to be qualified CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 307 Shouldnt Points Be Used? The several prior slides discussing scores (points) and grades from school are very clear Doesnt that prove that points should be used in evaluated procurements instead of just ranking? No! In school, almost all testing is objective;

e.g., Multiple choice, true false, fill-in-the-blank In testing, there is 1 right answer If a student got 90 out of 100 objective questions right, that 90 could accurately be compared to the scores of all other students Objective testing permits objective comparisons CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 308 Points Shouldnt Be Used

The evaluation of proposals is a subjective, not objective process It is more analogous to the Olympic scoring in gymnastics, figure skating, diving, etc. Judges assign the scores they feel are appropriate Hopefully, conscientiously But occasionally, points are assigned for other purposes; e.g., To help assure that a given competitor either wins or loses Think Russian judges during the Cold War CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 309 Points Shouldnt Be Used

The assignment of points in evaluated procurements do not make the evaluation objective It is still subjective There is no right or wrong answer with proposals Its how the individual evaluators thought each proposal should be scored In the distant past when points were routinely used, the first issue was deciding while the RFP was being written how many points to assign to each criterion, with the total equaling a given number, such as 100 points, 500, 1,000, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 310 Points Shouldnt Be Used

The most typically total was 100 points If there are 5 evaluation criteria, should the point breakdown be: 30, 25, 20, 15 and 10? 35, 25, 15,15 and 10? 25, 20, 20, 20 and 15, Etc.? Often, evaluators would agonize over the points to assign to each offeror in each evaluation criterion E.g., should proposal A receive 28 of 30 available points for criterion 1, versus proposal B getting 27 points, or 26? Frequently, there were 5 or more point differences between 2 given offerors, but evaluators would describe

these offerors as being essentially equal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 311 Points Shouldnt Be Used If each evaluator could readily have assigned 1 or 2 more points to a given offeror in each criteria, a 5 point difference could just as easily have been 4, 3, 2,1 or none. And, if points are assigned to the technical proposal, points also have to be assigned to the financial proposal Which means a point total had to be assigned when the RFP was written

And then a ratio was established between the technical & financial proposals, such as: 50/50; 60/40; 70/30; etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 312 Points Shouldnt Be Used The result was a twofold award formula 1. 2. While most of the time there was satisfaction with the award result

Much of the decision making was made in advance, while the RFP was being written, instead of after proposals were read and evaluated The point totals would dictate the winner of the procurement The offeror with the highest combined point total winning Sometimes, the evaluation committee would say they wished a different offeror would have won i.e., they were painted into a corner by the up-front points & ratio & having to assign points by criteria CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 313 Points Shouldnt Be Used

If they had the freedom (flexibility) to select whichever offeror they believed to be the best overall value, they would have selected a different offeror Also, the assigned points were often the target of protests Attorneys for protesters tried to get the evaluation committee members or procurement officer to admit they readily could have given a few more points to the protester And often a few points difference, offeror to offeror may have changed the outcome CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 314

Points Shouldnt Be Used It is much easier to identify which offeror is best, which is number 2, etc., for each evaluation criteria Than to assign a specific number of points to each given offeror for each evaluation criteria And also easier to say which offeror is believed to overall be the most advantageous Than to assign technical points and then open financial proposals and have a ratio (weight) determined when the RFP was written make the award decision Good or bad CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 315 Points Shouldnt Be Used Once points and ratios stopped being used in DHMH (MDH) in 1995 or 1996, evaluators quickly voiced their preference over the way awards previously had been done i.e., the points hindered instead of helped Now they were free to recommend the offeror they wanted to win rather than the one the formula said had to win CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 316

Example 1: Points Shouldnt Be Used In the mid-1980s most of what is now the Dept of the Environment was part of the DHMH Office of Environmental Programs (OEP) OEP did a procurement to obtain a consultant for some important environmental study The Asst. Sec. for Environmental Programs had a high interest in the procurement and placed himself on the evaluation committee Since this was pre-1995, points and weights (technical/financial ratio) were used At the end of the process the points and weights dictated an award to an offeror that was $50,000 higher than another quality offeror

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 317 Example 1: Points Shouldnt Be Used The OEP Asst. Secretary said the formula selected offeror wasnt $50,000 better than the other offeror He asked me if the award had to go to the higher priced offeror and essentially waste $50,000 I said, Yes. We had to follow the dictates of the formula written into the RFP Now this seems silly, but back then this process was carved in stone CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 318 Example 2: Points Shouldnt Be Used About 1990 the first ever procurement was done for a smoking cessation media campaign in Maryland There were 5-6 offerors, but 1 was unanimously judged by all members of the evaluation committee to be technically superior to all others The committee members were excited that this offeror would win the award because they felt this vendor would have a pronounced effect on reducing smoking in Maryland i.e., it had the ability to produce ads & articles and hold

events that would spur people to actually quit smoking, rather than ones that would be ignored CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 319 Example 2: Points Shouldnt Be Used Even though this offeror received excellent technical point totals, it also had a very high price The committee felt this offeror was worth every cent because of the belief it would achieve actual results When the prices were opened, another offeror that was OK technically had a very low price The formula dictated the award to this other offeror

The evaluation committee members were bitterly disappointed with the result because they felt the money that would still be paid under the contract would have little effect in getting smokers to quit i.e., paying more money to get results was better than less money with little to show for it CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 320 Example 2: Points Shouldnt Be Used The evaluation committee members, with my concurrence, even tried to persuade the DHMH

Deputy Secretary (agency head designee) to award to the preferred offeror Because the Deputy had the formula approach ingrained in his mind, he declined to alter the result Again, now it seems bizarre to make an award that the people involved in smoking cessation didnt believe would accomplish anything But, at that time this was accepted as the way it had to be CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 321 Formal Recommendation for Award Letter WHAT TO INCLUDE WHAT TO EXCLUDE CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

322 Recommendation for Award When the Evaluation Committee completes its work and has determined which offeror to recommend for the award A letter needs to be drafted to formally report this recommendation to the Procurement Officer and the agency head or designee This letter should contain the rationale for making the award to the recommended offeror in as much detail as deemed necessary CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 323

Recommendation for Award This letter should contain various sections, such as the following, not necessarily in the order in which they appear in these slides: 1. 2. A brief overview of the procurement A brief chronology, probably as an attachment, with dates when: The RFP was issued The Pre-proposal conference was held Answers to questions were provided, including the total number of questions received Any Addendums were issued Proposals were due, including the number received CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 324

Recommendation for Award This letter should contain various sections, such as: A brief chronology with dates when: The committee members met to discuss proposals Cure letters were sent and responses were received Orals were held Post oral presentation follow-up cures were received The committee formalized the technical rankings and opened financial proposals BAFOs were requested and received The committee finalized its overall rankings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 325

Recommendation for Award This letter should contain various sections, such as: 3. A summary of the competition The outreach or notification of the procurement, including eMM notice, web site listings and direct, e.g., email, notification to vendors Separately describe the MBE and Maryland vendors notified How many proposals were received How many proposals were deemed qualified and had financial proposals opened versus non-qualified If competition was less than expected, some explanation of why

E.g., Notice to Vendor responses of why didnt participate CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 326 Recommendation for Award Recommendation letter sections: 4. 5. The members of the evaluation committee, possibly including brief background information and perhaps why these persons were chosen Any notable occurrences, complications, etc. during the procurement, including some explanation if the procurement took significantly longer than expected, such as:

A protest against the specifications or proposed award Any action that was taken because of a delay, such as an extension of a current contract CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 327 Recommendation for Award Recommendation letter sections: 6. The primary reasons for recommending the selected offeror While it is alright to use qualitative descriptions for offerors proposals, such as Excellent, Very Good, etc. Using such labels alone is insufficient Explain the primary aspects of the proposals that

resulted in these labels, such as described in the several slides starting with slide 303 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 328 Recommendation for Award Recommendation letter sections: 7. An explanation of any efforts to validate that the recommended offeror was responsible, such as: 8. Internet search looking for negative newspaper articles, law suits, etc. that were not disclosed by the offeror in its proposal

A comparison of the proposed award to any current contract for the same activity, including: Any major differences or improvements Price comparisons; tout and explain savings or explain higher prices, especially if the award price is much more than unexpected CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 329 Recommendation for Award Recommendation letter sections: 9. 10.

11. A list of all attachments, with a brief description of what each depicts An offer to meet with or otherwise provide further information if requested to do so A formal request for the agency heads approval and how that approval should be indicated; e.g., Write approved and sign and return a copy of the recommendation letter Send a separate approval memo Send an email stating approval CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 330

Recommendation for Award The letter should include Attachments of: A list of all proposals received, including ones deemed not qualified With offeror name and city and state where located A chart showing the overall final technical rankings of all qualified offerors Including the ranking for each evaluation criteria A chart showing offerors final prices A chart showing the overall (combined technical and financial) rankings for each offeror Either a chart or narrative of each offerors major strengths and weaknesses

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 331 Recommendation for Award The letter should not: Be excessively long with lots of minor information The main body of the letter likely should be only 3-5 pages Most of the detail and volume should be in the attachments If the agency head wants the detail he/she can look at all the attachments, or just those interested in If the agency head just wants the essentials, he/she can get that from the main body

i.e., he/she doesnt have to wade through pages of charts and details in the letter itself just to identify the recommendation and what is being asked of him/her CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 332 Recommendation for Award The letter should not include: Excessively negative or positive descriptions of any offeror with wording that might indicate bias either for or against any offeror Unsubstantiated allegations against any offeror Grammatical or spelling errors, or narrative that contradicts attachment information Contradictory information vs. information provided in debriefings, a protest response, answers to questions, etc.

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 333 Recommendation for Award If there is a protest against the proposed contract award, frequently the attorney representing a protester will request a copy of this letter Usually, if the protester attorney agrees to confidentiality, the Board of Contract Appeals will require disclosure of this letter The protester attorney would then look for information in this letter that contradicts anything previously told/disclosed to the protester Which would be used to augment the protest

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 334 Recommendation for Award This letter should be routed through the: Contract Officer who should initial the letter to indicate agreement with its content and recommendation Procurement Officer who can Sign-off on it to indicate agreement with its content and recommendation and then forward to the agency head or designee for approval or disapproval; or, Remand the letter for changes, corrections, greater justification for the recommendation, etc. Reject the recommendation and make his/her own award decision, including sending his/her own

recommendation letter to the agency head CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 335 Procedures for Informing Unsuccessful Offerors under Competitive Sealed Proposals Procurements What to Say! When to Say It! How to Say It! 336 The Sooner, the Better The sooner an Offeror is notified in writing it is unsuccessful, the sooner: A debriefing can occur A protest can be filed

A protest can be answered The period for filing a protest expires CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 337 Prompt Written Notification to an Offeror it is Unsuccessful Allows timely opportunity for the Offeror to try to refute the determination A mistake may have been made in eliminating an Offeror If an Offeror convinces the procurement officer it was improperly eliminated: The elimination can be rescinded The Offeror can be placed back into consideration for the award

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 338 Provide all Disclosable Information Together Only give one bite at the apple The 7-day protest period starts over each time an Offeror is provided with new substantive information; i.e., What its technical ranking was What total prices were (not unit prices) Who won Detailed evaluation information provided during a debriefing CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 339 Provide all Disclosable Information Together If initially an Offeror is only told it didnt win And later the Offeror is told: Who submitted proposals What the technical rankings were What the total prices were (not unit prices) What the overall rankings were Detailed deficiencies of its proposal as is provided during a debriefing Then the protest period may start again CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 340 Types of Unsuccessful Offerors The Proposal is determined to be: Not Reasonably Susceptible of Being Selected for Award Reasonably Susceptible of Being Selected for Award It is not technically acceptable

OR But not determined to be the Most Advantageous Offer The Offeror is deemed not to be responsible CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 341 Responsibility Characteristic of the Offeror itself and not the proposal Offeror is determined to lack one or more of: Capability Integrity Reliability

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 342 Proposal not Technically Acceptable (Not Reasonably Susceptible of Being Selected for Award) General preference is to not eliminate Offerors until after discussions are held Offerors should have an opportunity to: Clarify their proposals Address identified proposal deficiencies Try to prove they are responsible

But an unacceptable determination can be made any time after proposals are received and read CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 343 Proposal not Technically Acceptable (Not Reasonably Susceptible of Being Selected for Award) Notify an Offeror in writing that its proposal is deemed unacceptable as soon as this determination is officially made CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 344 Notify of Unacceptable

Proposal Status If this determination is made before the final award recommendation is made: Include a brief rationale for the determination Offer to promptly debrief the Offeror Explain if there is nothing more to say about the rejection than what was stated in the rejection notice But if the offeror still insists on a debriefing, one must be provided, even if what was in the rejection notice is simply repeated CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 345 Notify of Unacceptable Proposal Status (Contd)

If this determination is made at the same time as the final award recommendation Include a brief rationale for the determination Offer to debrief the Offeror Provide the: Register of Proposals Final technical rankings Final total financial proposal prices (not unit prices) Final overall rankings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 346 Offeror Deemed Not Responsible If this determination is made before the final award recommendation is made:

Include a brief rationale for the determination Offer to promptly debrief the Offeror Explain if there will be nothing more to say about the rejection than what was stated in the rejection notice But if the offeror still insists on a debriefing, one must be provided, even if what was in the rejection notice is simply repeated CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 347 Offeror Deemed Not Responsible (Contd) If this determination is made at the same time as the final award recommendation: Include a brief rationale for the determination Offer to promptly debrief the Offeror

Provide the: Register of Proposals Final technical rankings Final total financial proposal prices (not unit prices) Final overall rankings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 348 Not Most Advantageous Offer Can only make this determination when the recommendation for award is made, so: Offer to debrief the Offeror Provide the: Register of Proposals

Final technical rankings Final total financial proposal prices (not unit prices) Final overall rankings May give a brief rationale for the technical rank But cant disclose information about other proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 349 Recommendation for Award When an agency officially determines the Offeror to which it will offer a contract for the

required service This is now the time when: The notification to all other Offerors described in the prior slides in this section is permitted Non-Offerors can receive: The Register of Proposals The identity of the Offeror being recommended for the contract award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 350 Recommendation for Award (Contd) Usually official notification of the award recommendation to non-selected offerors

should wait until the selected Offeror has satisfied pre-award RFP requirements before announcing it is the recommended Examples of possible requirements that the recommended offeror must satisfy It is accepted as having made a good faith effort to meet the MBE goal This may take 10 working days to determine after initial notification, including if there is a waiver request CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 351 Recommendation for Award (Contd) It has submitted: A performance bond or other type of

contract/performance surety, if applicable Any required evidence of adequate insurance The Contract Affidavit The contract has been signed as submitted by the agency The Comptrollers Office has certified: There is no outstanding debts or tax or corporate registration deficiencies CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 352 Recommendation for Award (Contd)

Urge the recommended offeror to submit all required information as soon as possible, but not later than as stated in the RFP It can be a serious problem if prices and other information has been divulged to nonwinning offerors when a particular Offeror is identified as being recommended for award but the award selection has to be rescinded because of the recommended offerors failure to satisfy pre-award RFP requirements CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 353 Debriefings CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 354

DEBRIEFING A debriefing is the sharing of information with an unsuccessful offeror by a procurement official familiar with the rationale for the selection decision. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 355 AWARD RECOMMENDATION (Contd.) DEBRIEFING The purpose of the debriefing is to allow the offeror an opportunity to learn how it can improve its future submissions This information is limited to the strengths and weaknesses of that offerors current proposal. The information provided in the debriefing must be limited to a discussion of the unsuccessful offerors proposal. CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 356 AWARD RECOMMENDATION (Contd.) DEBRIEFING The debriefing should occur at the earliest feasible time after the Procurement Officer makes the final determination recommending the award i.e., before an award is finalized, in order to allow an unsuccessful offeror sufficient time, should they choose, to file a protest and have it resolved before the award is finalized. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 357 AWARD RECOMMENDATION (Contd.) DEBRIEFING Specific discussion of a competing offerors proposal is prohibited. The debriefing must be factual and consistent with the

evaluation of the unsuccessful offerors proposal. The debriefing must provide information on areas in which the unsuccessful offerors proposal was deemed weak or deficient. Discussion or dissemination of the thoughts, notes, or rankings of individual members of an evaluation committee is prohibited. A summary of the procurement officers rationale for the selection decision and recommended contract award can be included. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 358 The Debriefing Session The Offeror typically has the right to choose the manner of the debriefing Usually a debriefing is conducted in person But, an Offeror can be debriefed by phone, or possibly via other means

Be prepared: Have all strengths & weaknesses grouped by the evaluation criteria stated in the RFP All points must relate to the evaluation criteria identified within the RFP CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 359 The Debriefing Session (Contd) Limit the number of State personnel But have appropriate personnel present Usually just the procurement officer and one (the most appropriate) member of the evaluation committee should conduct the session Include an AAG if an offerors attorney will participate (See slide 371)

Plan to tactfully discuss/focus on: The weak/deficient areas of the proposal; or, if applicable: Why the Offeror was deemed not responsible CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 360 The Debriefing Session (Contd) Its probably best to have a pre-prepared script of the most pertinent points you plan to discuss & give the offeror a copy of the script at the beginning of the session

If you dont do this but have any written notes, the offeror will probably ask for a copy anyway Having a script potentially saves time by: Providing the names & titles of State personnel present Listing the ground rules for the session Minimizing note taking by the offeror CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 361 The Debriefing Session (Contd) Reading from a prepared script, with the offeror reading along, should reduce: State personnel fumbling for what to say, how to say it and perhaps straying off subject Offeror questions

While the offeror can ask questions about the information being provided A focused presentation should allow the session to be completed faster, which is a good thing As long as the offeror is provided with the information to which it is entitled, the sooner the session ends the better CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 362 The Debriefing Session (Contd) Ensure the other State personnel present have been pre-instructed to avoid going offscript with spontaneous comments

Especially comments that provide more information than was asked or that venture into tangential issues At the very beginning of the session: Make introductions by name and title The script will have this information for State personnel Someone for the State, usually the procurement officer, should then cover all the ground rules CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 363

The Debriefing Session (Contd) The ground rules are the limitations on what can and cannot be discussed, such as: This is not a forum to re-open the evaluation There will be no debate The points being discussed are the conclusions of the evaluation committee & P.O. findings However, the offeror can raise issues if it feels that there are inaccuracies in what it is being told concerning its proposal Essentially the points listed on Slides 358 & 365: No discussion of competing offerors proposals No divulging of comments, etc. of individual members of evaluation committee

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 364 The Debriefing Session (contd) Information provided must be limited to the unsuccessful Offeror's proposal Information from a competing Offeror's proposal is not to be disclosed Discussion or dissemination of the thoughts, notes, or rankings of individual members of an Evaluation Committee is prohibited A summary of the Procurement Officer's rationale for the selection decision and the

recommended contract award can be included CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 365 The Debriefing Session (Contd) At this point this is the official Dept. position Both the Procurement Officer and Agency Head or designee must have already approved the recommendation for award Which only they, not an evaluation committee, have the authority to do Even in an extreme situation such as the P.O.

recommending a different offeror for the award, such information should not be divulged It is probably discoverable later if there is a protest, via the award recommendation letter discussed starting on Slide 322, but not now CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 366 The Debriefing Session (Contd) Even in the overwhelmingly typical situation of the evaluation committee recommendation being accepted by the P.O. & Agency Head, all information provided should be attributable to the evaluation committee as a whole Do not indicate disagreement among evaluation

committee members, such as: It was a 3 to 2 vote on the recommended offeror or how a given criteria was ranked; or, 1 member felt strongly that . CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 367 The Debriefing Session (contd) Do not talk too much Do not debate the Offeror This is not a forum to re-open the evaluation If the offeror wants to have prolonged discussion on a point that the P.O. believes has been discussed to the degree possible or appropriate, say so and Move On!

If the offeror seems satisfied on a given point with the information provided, dont prolong the discussion; Move On! CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 368 The Debriefing Session (contd) For a good offeror that was not chosen as the most advantageous offeror, there may not be a lot to say other than another offeror was determined to have submitted a better overall proposal This is especially true if the offeror being debriefed

was better, or even best, technically but was much higher in price In this situation about all that can be said is the offeror lost on price But dont make it appear that price had more impact than the weight described in the RFP E.g., equal with technical, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 369 The Debriefing Session (contd) For an offeror without a lot of weaknesses

and with a reasonable price You may simply have to say that despite the lack of weaknesses, the debriefed offeror can infer that higher ranked offerors, including the selected offeror, had fewer weaknesses, or more or greater strengths And adhering to the prohibition on discussing another offerors proposal Dont provide any details about the other proposal(s), even if asked for examples CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 370 The Debriefing Session (contd) When scheduling a debriefing inquire of the anticipated attendees, including those who will be listening on a speaker phone If it is indicated that the offeror will have an

attorney present, say that you must also have an AAG present Sometimes the offeror will then say they wont have their attorney involved Otherwise, try to only schedule the debriefing when an AAG can participate CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 371 Notify Offerors Before Submitting for Approval Before submitting contracts to a control agency for approval (including approval by the BPW): Unsuccessful Offerors must be offered a debriefing It is preferred that debriefings actually occur

prior to the approval submission This saves an item having to be pulled from a BPW agenda if a protest is received before the award date CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 372 Notify Offerors Before Submitting An approval request can be submitted to a control agency after a reasonable amount of time has expired since Offerors were notified of the opportunity for a debriefing Generally 7-10 days Offerors that dont request a debriefing within this time period are considered not to be interested in being debriefed

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 373 Notify Offerors Before Submitting (Contd) An approval request can also be submitted to a control agency if an Offeror requests a debriefing within this time period but states it cannot participate in a debriefing until considerably later than the offered 7-10 day debriefing period But the Offeror should be notified that the approval request is being submitted due to its inability to schedule a timely debriefing CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

374 Delayed Debriefings Requests Occasionally, an unsuccessful offeror may not respond to a debriefing offer within the indicated 710 day timeframe E.g., it might wait 2-3 weeks, or even longer before asking for a debriefing Or, wait until after an award has been officially approved While there is no guidance on what is a reasonable timeframe for a debriefing request Usually it is best to give the benefit of any doubt to

the offeror and hold the debriefing In such an instance the offeror may genuinely be trying to learn how to do better the next time CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 375 The Steps in the CSP Evaluation Process From Approximately the Time Proposals are Received Until Completion of the Procurement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 376 CSP Evaluation and Selection Process 1.

2. Make appointments to committee Get acceptance of appointment 3. Certification of impartiality Agreement to maintain confidentiality Distribute RFP & pertinent documents Addendum Questions from prospective offerors & answers Proposal evaluation form CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 377 CSP Evaluation and Selection Process 4. Procurement Officer (PO) meets with the evaluation committee to discuss Operating procedures 5. What to do What not to do

Review documents Evaluation Committee Duties & Responsibilities & Guidelines for Instructions to an Evaluation Committee Projected timeframes and scheduling Evaluation criteria Receipt of proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 378 CSP Evaluation and Selection Process 6. Initial review of technical proposals by P.O. for: Compliance with any mandatory

requirements No inclusion of cost information Obvious omissions Exceptions to RFP CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 379 CSP Evaluation and Selection Process 7. Possible P.O. contact with offerors about Evidence that meet mandatories Removal of price information Submission of incomplete or omitted items Required additional details or corrections

concerning information on affidavits, such as: Except for missing MBE forms which is not curable Admission of past bribery conviction Wrong resident agent Required additional information about law suits or financial statements based upon information provided in the proposal or otherwise known CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 380 CSP Evaluation and Selection Process 7. Possible P.O. contact with offerors (Contd) Confusion of the correct legal name of the

offeror, or who actually is the offeror Issue of parent firm vs. sister firm vs. subsidiary There may be different names for the offeror in different places in the proposal, such as: CSP - 4/2018 Insurance form Bonds Financial Statements SEC corporate filing Transmittal letter References Legal proceedings Copyright 2018 Md. Dept. of Health - Unpublished Work

381 CSP Evaluation and Selection Process 7. Possible P.O. contact with offerors (Contd) Clarification should be sought on any difference in names, no matter how slight Many companies have subsidiaries with slightly different names, each being a separate legal entity with separate assets and governance Often this is done to remove legal and financial liability (risk) from the parent (holding) company We need to know if the actual offeror is essentially a hollow shell that can easily cease to exist, versus a substantial firm

To determine whether to seek a parental guarantee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 382 CSP Evaluation and Selection Process 8. Rejection of: An offeror if it is determined not to be responsible for reasons such as: Does not satisfy Minimum Offeror Requirements Is debarred Has some other situation where a non-responsible determination can be made at this early stage

An offerors proposal if an offeror failed to properly complete and submit MBE forms See step 12 (slide 386) for additional information concerning a rejection CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 383 CSP Evaluation and Selection Process Distribution of proposals to committee, except for proposals for offerors already rejected, per step 8 9. Also dont distribute the proposal of any offeror for which rejection may still occur

10. If still dont have sufficient information to make a definite rejection determination but need to proceed pending outcome Committee reads proposals in the same order if there are too many proposals to read and discuss at one meeting CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 384 CSP Evaluation and Selection Process 10. Committee discusses proposals

11. All members should participate in these discussions Best if take place in person Can take more than one meeting Committee & P.O. discuss whether to: Amend the RFP based upon proposal responses Eliminate any offeror at this time because its proposal is judged so deficient that it cannot be improved with discussions or if the offeror is judged incapable of performing requirements CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 385 CSP Evaluation and Selection Process

12. If P.O. agrees to eliminate an offeror based upon technical deficiencies or responsibility issues, the P.O. sends elimination letter & unopened financials Include a brief rationale for the elimination Include an invitation for a debriefing, although there may be little more to say than what was included in the elimination letter An offeror which is eliminated for technical deficiencies, or as not being responsible can be debriefed immediately, before a decision is made concerning a recommendation of award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 386

CSP Evaluation and Selection Process 12. (contd.) Once an offeror is eliminated there is the potential for a protest(s) If a protest is received, the P.O., with consultation from an A.A.G. & possibly the evaluation committee, should decide whether to halt the process until the protest is resolved, or continue CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 387 CSP Evaluation and Selection Process The Committee identifies questions or cure issues to be sent to each offeror

13. Have the offeror clarify any wording that is unclear Tactfully tell offerors unacceptable content or content that will be evaluated negatively Dont try to guess what a proposal is saying Such as personnel judged as not qualified Provide sufficient time for adequate responses

If at all possible, have the responses due before orals so they can be discussed there if deemed appropriate CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 388 CSP Evaluation and Selection Process 14. The cure responses are received and reviewed by the P.O. and Committee A decision is made of whether there is both a need for, and time for follow-ups to the cure responses prior to the orals If so, additional cure letters are distributed If not, proceed to step 15.

Caution: if a follow-up cure response is deemed necessary for one or a few offerors, but not all, consult with an AAG about the permissibility of not sending some type of follow-up to all offerors Or, perhaps skip follow-ups and deal with remaining cure issues at the orals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 389 CSP Evaluation and Selection Process 15. Committee creates questions for offerors at oral presentation Create generic questions for all offerors Asking each offeror the same generic

questions: Creates a sort of baseline of offeror responses that should be comparable Can be used to spur discussion for offerors that arent talking much Can fill time for offerors for which there arent a lot of specific questions, such as an incumbent CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 390 CSP Evaluation and Selection Process 15.

(Contd) Create specific questions for each offeror Dont leave any substantive issue unquestioned or undiscussed Within reason, make sure each offeror is informed of substantive concerns, perceived weaknesses, lingering issues from the cure responses, etc. An unselected offeror should not learn of a substantive weakness at a debriefing Bring it up while there might still be potential for improvement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 391 CSP Evaluation and Selection Process 16. (Contd) Committee & P.O. decide on general

format of orals: Where the orals will be held, with consideration for Parking Maximum number of attendees Actual seating arrangement CSP - 4/2018 So everyone can see and hear So evaluators can take notes without offeror personnel being able to read

Noise/distraction levels Availability of a waiting area if offerors attend early or a prior session runs over Availability of a greeter Copyright 2018 Md. Dept. of Health - Unpublished Work 392 CSP Evaluation and Selection Process 16. Concerning orals, committee & P.O. decide: When orals will be held What is a reasonable length of advanced notice to offerors (# of days notice) Leeway to accommodate offeror preferences Actual schedule of orals in terms of allowing time for:

Evaluators to eat, use rest rooms and discuss one offeror before having an oral with the next A time gap so an offeror leaving doesnt encounter one arriving A presentation session running a little longer than anticipated CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 393 CSP Evaluation and Selection Process 16. Concerning orals, committee & P.O. decide on (Contd)

Duration of orals Should be scheduled for at least 1 hour May allow 2-3 hours for complicated RFPs Any recommended or required offeror personnel to attend the orals or limitation on number of offeror attendees Whether to have the orals transcribed Whether to have any technical advisors present to help the committee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 394

CSP Evaluation and Selection Process 16. Decide on general format of orals: How much time for offerors to make introductions and opening remarks Usually 15 minutes if the session is 1 hour and maybe 30 minutes if 90 minutes or more The emphasize should be getting to the generic and specific questions, not extensive sales pitch Any prohibitions on elaborate presentations If a product demonstration is required If there will be internet or phone access and if so, where it will be within the room

CSP - 4/2018 If access is in remote corner it may not be useful Copyright 2018 Md. Dept. of Health - Unpublished Work 395 CSP Evaluation and Selection Process 16. (Contd) Decide on general format of orals: Whether a slide projector will be made available or if offerors can bring one The number of copies of handouts to bring How much time to permit after the orals for offerors to respond to follow-up issues One for each evaluator, the P.O. and maybe

others E.g., 3 days, 5 days, etc. Which State personnel will record each issue identified as needing follow-up answers to compare with offeror notes after the orals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 396 CSP Evaluation and Selection Process 16. (Contd) Decide on general format of orals: How evaluators will ask questions (one person ask all, do round-robin, etc.), including follow-up ones Who will control the meeting from the State perspective to:

Make introductory remarks Give ground rules to offerors Inform them they must record follow-up issues to reconcile with the State person doing the same Keep offerors on required time line Keep offerors on subject, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 397 CSP Evaluation and Selection Process 16. (Contd) When discussing the format of the orals, the P.O. should also stress to the evaluators their role and responsibility

i.e., they need to speak-up if there is anything they hear that: Is confusing or wrong Seems contradictory to what was in the offerors proposal or prior cures Is new information that needs a written follow-up The oral session is their best opportunity to gauge the commitment of senior officials, capability of the offerors personnel, fluidness of interaction with subcontractors, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

398 CSP Evaluation and Selection Process 16. Offerors that are technically acceptable or potentially so are contacted & scheduled for oral presentations: Provide notice of all times, dates, ground rules, etc. agreed to in steps 14 & 15, and possibly some of the questions developed in step 13, if not previously provided or if additional discussion is deemed necessary Ensure offerors understand ground rules in terms of whom to bring, type of presentation to do or not do, duration, that they will need to take notes of follow-up items, etc. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

399 CSP Evaluation and Selection Process 16. (Contd) Provide the exact location for the orals, directions, including directions within a campus or building, and any special instructions, such as parking instructions or the need to sign-in at a security desk and get passes, or be escorted Provide contact information in case something goes wrong and an offeror needs to reach someone at the last minute

Alert offerors to anything that might take time and potentially delay their arrival for their oral session Preferably someone assisting the Eval. Comm. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 400 CSP Evaluation and Selection Process 17. Oral presentations are held with offerors Any offeror exceptions should be discussed Offerors should be put on notice if exceptions have the potential to preclude an award But, exceptions do not have to be conclusively dealt with at this time

If an exception is to a mandatory clause, it should be emphasized that there is no authority to waive it Although some mandatory clauses allow minor re-wording It should also be emphasized that except for very minor revisions, that agreement to an exception cannot be made just for 1 offeror CSP - 4/2018 An amendment must be done to the RFP Copyright 2018 Md. Dept. of Health - Unpublished Work 401 CSP Evaluation and Selection Process (Contd) Stress, that because an amendment is needed,

making the revision available to all offerors still under award consideration, there is a general reluctance to do this Also, stress that an amendment at this time cannot fundamentally change the specifications such that if the change had been in the original specifications that vendors that did not submit a proposal might have submitted one If because of all the above the offeror volunteers to remove the exception, good 17. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 402 CSP Evaluation and Selection Process

17. (Contd) If at the end of the procurement a vendor with exceptions isnt selected for the award, dealing with the exception never would have been necessary But, sometimes when you get to the end of the process, it turns out that all prices are unaffordable or an offeror with exceptions is by far the best choice You may be willing to accept a change at this time that you previously wouldnt accept to get lower prices or to keep a superior offeror in the running Generally, save remove exceptions or be eliminated ultimatums until the end of the process when you are sure this is what you want CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

403 CSP Evaluation and Selection Process 17. Committee holds oral presentations with offerors All committee members should attend all orals P.O. or Contract Officer should attend Keep the session on focus, addressing the: Previously drafted specific questions Previously drafted generic questions if they have not already been covered with the answers to the specific questions or the offerors introductory presentation Any cure issues still outstanding

i.e., if the response to a previously identified cure issue is not satisfactory CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 404 CSP Evaluation and Selection Process 17. (Contd) At the end of the session coordinate any required additional follow-up Compare the follow-up notes taken by the Offeror to the notes taken by the Evaluation Committee to ensure agreement on what is needed Agree on a response timeframe usually 3-5 days depending upon the number of issues to be addressed Offerors should be permitted about the same duration of time to respond

If possible, have the exiting offeror leave by a route that wont have them encounter the next presenting offeror, if any CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 405 CSP Evaluation and Selection Process Committee discusses performance of offerors at orals 18. Usually it is best to do this while the impressions and information from the orals are fresh, such as:

Between each oral if there is time At the end of the day Or, could: Have a later separate meeting Including waiting for all follow-up submissions Committee decides whether to recommend elimination of any offeror at this time 19. After receiving any follow-up information CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 406

CSP Evaluation and Selection Process P.O. decides whether to agree with any elimination recommendation 20. If agrees, as per step 12, sends elimination letter(s) & unopened financial proposals Letter includes debriefing opportunity notice Any elimination again brings the possibility of a protest P.O. decides whether any addendum is needed to the RFP, possibly with input from the evaluation committee 21. But, fundamental changes are not permitted

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 407 CSP Evaluation and Selection Process 22. If an amendment has been issued, P.O. & Committee discuss whether to invite a technical proposal best & final offer (BAFO) based upon the contents of the addendum Agree to scope of any technical BAFO Agree whether to permit revised sealed financial proposals with technical BAFO If no BAFO is requested, offerors must be required to acknowledge receipt of the addendum CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 408 CSP Evaluation and Selection Process 23. If no BAFO is requested: Committee agrees on final offeror technical pass/fail recommendations Committee agrees on final technical ranking of offerors Committee creates formal written summary of rationale for technical rankings

Committee submits technical recommendations to P.O. P.O. decides whether to accept the technical recommendations of the committee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 409 CSP Evaluation and Selection Process For any offeror that did not make final technical pass cut, P.O. sends: 24. Elimination letter with right to debriefing notice Unopened financial proposals Offerors eliminated at this point may protest Once final technical rankings are assigned,

financial proposals are opened for all technically qualified offerors 25. In rare instances there might be separate technical and financial proposal review committees. The above step assumes the typical situation where the same committee reviews both proposals CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 410 CSP Evaluation and Selection Process 25. (Contd.) If there are separate committees, financials would have been opened at the same time as technical proposals but the separate committees may not communicate with each other. Neither committee will

know what the other is doing or what it has found or determined, except the financial committee will be informed if an offeror has been eliminated as not being technically qualified. To use separate committees the RFP either must have specifically anticipated this, or at least must not have described an evaluation process that is counter to this. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 411 CSP Evaluation and Selection Process 26. If invite BAFO: Waits until offeror BAFOs received Committee & P.O. review BAFOs

P.O. decides if additional technical BAFO or changes to RFP should occur, possibly with input from the evaluation committee If no additional BAFO is desired conducts procedure from step 23 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 412 CSP Evaluation and Selection Process 27. If additional BAFO or addendum is done: Waits until offeror BAFOs are received Committee & P.O. review BAFOs P.O. decides if additional technical BAFO

or changes to RFP should occur, possibly with input from evaluation committees If no additional BAFO is requested conducts procedure from step 23 (There must be written approval from the agency head or designee to request any BAFO after the first one, including any BAFO on technical issues only.) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 413 CSP Evaluation and Selection Process 28. 29. Once final technical rankings are assigned financial proposals are opened for all technically qualified offerors Once financial proposals are opened, although rarely done, a decision is made on

whether discussions are needed on offerors financial offers Having financial discussions adds complexity since may only have issues with 1 offeror But if discuss with 1, have to discuss with all CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 414 CSP Evaluation and Selection Process 29. (Contd) Examples of issues where discussions might be beneficial are:

How can the offeror do everything offered for its price Price too low Now that see the cost of certain components, can say they arent worth it Just plain too expensive Discovered exceptions to the RFP that had not previously been disclosed CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 415 CSP Evaluation and Selection Process

29. Examples of issues where discussions might be beneficial are: (Contd) The price was conditioned on the occurrence of some other circumstance Apparent confusion on how the financial proposal was to be completed Financial proposal form was completed differently than required, including offeror substituting a different version of the form CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 416 CSP Evaluation and Selection Process

30. Could even decide to eliminate an offeror at this stage if: 31. The financial proposal reveals serious technical deficiencies that could not have been detected earlier At this stage any discussions or eliminations are very tricky Typically should be done only after consultation with A.A.G. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 417

CSP Evaluation and Selection Process Can decide to invite financial BAFOs After receiving a BAFO can decide whether to invite another one 32. 33. With written agency head approval After receive final BAFO 34. Do financial ranking Decide on overall rankings

Combination of technical and financial rankings Recommend an offeror for the award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 418 CSP Evaluation and Selection Process P.O. decides whether to accept the final award recommendation of committee 35. Since there should have been continuous participation/guidance by P.O. throughout the evaluation process, a committee recommendation is almost always accepted However, the P.O. does not have to accept the recommendation

Can remand back to committee for re-consideration Can make own decision with concurrence of agency head or designee Decision of P.O. and agency head is final decision of agency CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 419 CSP Evaluation and Selection Process Now any outstanding exception by the offeror recommended for the award has to be dealt with 36.

If the offeror refuses to eliminate an unacceptable exception, it can still have the award offer rescinded If the agency decides to accept an offerors exception, except for the unlikely event that the exception is negligible, all other offerors must be given the opportunity to enjoy the benefit of the exception Which means an addendum must be done Which must be followed by another BAFO request CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 420 CSP Evaluation and Selection Process (Contd)

36. After each BAFO there must be a re-evaluation by the Committee If another offeror submitted a much better BAFO this time versus any previous BAFO, the Committee can decide that offeror has now submitted the most advantageous offer and should be recommended for the award The offeror originally selected for the award should be informed of this situation Which may prompt that offeror to rescind any exception Which means an amendment and BAFO is not needed and the award finalization can occur CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 421 CSP Evaluation and Selection Process

(Contd) 36. But, if the selected offeror still takes one or more exceptions even after being informed that it may not be the selected offeror if an amendment is issued and another BAFO requested Proceed with the amendment and BAFO request Written agency head or designee approval will be needed if this is a 2nd or later BAFO CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 422

CSP Evaluation and Selection Process When the BAFO responses are received there must be a re-evaluation by the Committee Maybe the originally recommended offeror will again be recommended Or, maybe a different offeror will now be deemed to have submitted the most advantageous offer The P.O. must agree with the recommendation The agency head or designee must also approve the recommended award 37. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

423 CSP Evaluation and Selection Process Obtain a Comptrollers Office clearance indicating that the selected offeror is in Good Standing in terms of: 38. Having paid its taxes and other fees Being registered to do business in Maryland Check that the offeror is not debarred If the selected offeror is not debarred and is in Good Standing, notify it of its award recommendation and ask it to

sign the contract CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 424 CSP Evaluation and Selection Process There should be no further negotiations or significant changes to the final technical or financial proposals or the contract once an award recommendation is made 38. The selected offeror should be directed to sign the States contract without exception CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

425 CSP Evaluation and Selection Process 38. The selected offeror will be directed to submit any needed documents, such as: MBE participation forms, and/or waiver request VSBE forms Contract Affidavit Insurance binder Performance bond Non-disclosure forms Hiring Agreement Forms required if there is federal

funding CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 426 CSP Evaluation and Selection Process Usually, a general announcement of the award should be delayed until all required forms and the signed contract have been received When all required documents are received: 38. Release the register of proposals

Notify all other offerors of the award recommendation A list of all offerors who submitted proposals Include technical rankings, prices, and overall rankings Offer debriefings for all unsuccessful offerors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 427 CSP Evaluation and Selection Process If the recommended offeror 39. Cannot/will not submit all required documents

within the required timeframe; e.g., Wont sign the contract without changes Cant get a performance bond or insurance Wont sign the non-disclosure forms Doesnt provide the MBE Participation Forms Requests a MBE waiver which is not granted The award recommendation must be rescinded and that offeror eliminated Step 38 then commences again with the next highest ranked offeror CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 428 CSP Evaluation and Selection Process Debriefings of unsuccessful offerors should typically be conducted before submitting items for approval by a control agency or the BPW in case: 39. An offeror can convince the P.O. that it was mistakenly not selected A protest is filed within 7 days thereafter If a protest is filed the award cannot be completed until either the protest is resolved or approval by the BPW is requested notwithstanding the protest

BPW approval should only be requested if there is an extraordinarily compelling reason why an award needs to be made immediately CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 429 CSP Evaluation and Selection Process If there is an unresolved protest at any point during the procurement: 40. E.g., if an offeror is eliminated immediately

after proposals are opened for the failure to submit required MBE forms Final award approval cannot be obtained until the protest is resolved; or, The BPW approves the award notwithstanding the protest Which rarely happens CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 430 Guidelines for Making a CSP Award Recommendation CSP Evaluation Guidelines In CSP can make the award to the highest technically ranked offeror, or

The lowest priced offeror, or Any other offeror that is determined to be the best value in terms of what they will provide for the price they will charge CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 432 Which Would You Choose? #1 (Technical worth more than financial) Vendor Technical Price Rank Quality % Price Selected Variance Variance Vendor? A 1 350,000 Much

0 better #B B 2 290,000 Much 17% better #C less C 3 310,000 Slight 11% better #D less D 4 270,000

CSP - 4/2018 23% less Copyright 2018 Md. Dept. of Health - Unpublished Work 433 Which Would You Choose? #1 (Technical worth more than financial) Vendor Technical Price Rank Quality % Price Selected Variance Variance Vendor? A 1 350,000 Much 0 better #B

Probably B 2 290,000 Much 17% better #C less Maybe C 3 310,000 Slight 11% better #D less No D

4 270,000 Not likely CSP - 4/2018 23% less Copyright 2018 Md. Dept. of Health - Unpublished Work 434 Reasons For Choice Example # 1 A = probable awardee A = Highest technical rank & highest cost

Much better technically than B Much, much better technically than C or D Very possible that technical superiority is worth 17% higher cost than B Probable that technical superiority is also worth the 11% higher cost than C or the 23% higher cost than D CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 435 Reasons For Choice Example # 1 B = could be best value Not as good technically as A But 17% lower in cost than A Could decide that A technical superiority isnt

worth this 17% differential Much better technically than C or D & lower cost than C Not much more expensive than D CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 436 Reasons For Choice Example # 1 C = Would not award Not close technically to A or B Whereas is less expensive than A Is not less expensive than B No reason to select over B Not much better technically than D Probably wouldnt select over D

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 437 Reasons For Choice Example # 1 D = Lowest cost However, not much lower than B Also technically B is much better For price difference, would not select over B and maybe not over A CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 438

Which Would You Choose? #2 (Technical worth more than financial) Vendor Technical Price Rank Quality % Price Selected Variance Variance Vendor? A 1 350,000 Almost tie w/B 0 B 2 290,000 Almost tie w/C

17% less C 3 310,000 Almost tie w/D 11% less D 4 270,000 23% less CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 439 Which Would You Choose? #2 (Technical worth more than financial) Vendor Technical Price Rank Quality % Price Selected Variance Variance Vendor? A 1 350,000 Almost tie w/B 0 No B

2 290,000 Almost tie w/C 17% less Maybe C 3 310,000 Almost tie w/D 11% less Not likely D

4 270,000 23% less Maybe CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 440 Reasons For Choice Example # 2 Probably would not award to A A = Highest technical rank & highest cost Not much better technically than B, C or D Unlikely that technical difference is worth

the higher cost over either of B or D CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 441 Reasons For Choice Example # 2 B = could be best value Almost as good technically as A But 17% lower in cost than A Could decide that A technical superiority isnt worth this 17% differential But also not much better technically than C or D May not be worth extra cost over D CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

442 Reasons For Choice Example # 2 C = Would not award Close technically to A or B Whereas is less expensive than A Is not less expensive than B No reason to select over B Not much better technically than D Probably wouldnt select over D CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 443 Reasons For Choice Example # 2

D = Lowest cost Technically close to all of A, B & C However, not much lower in cost to B May decide B is better value But may also decide that for the slight technical difference that lowest cost represents the best value CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 444 What Would You Do Exercises CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

445 What Would You Do? # 1 Offeror comes in with equipment for a fancy slide show with graphics and sound & slick handouts And announces that its planned presentation will take an hour or more CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 446 What Would You Do? # 1 Situation: Offeror comes in with fancy slide show with graphics and sound & slick handouts

When discussions were scheduled should have already told offerors not to do this Should reiterate this & tell the offeror it may not put on such a show Since it was told not to do this, could ask if this is the typical manner in which this offeror follows directions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 447 What Would You Do? # 1 Situation: Offeror comes in with fancy slide show with graphics and sound & slick handouts Need to emphasize that the committees questions are more important that this sales pitch so need to

get on with the questions & get through them State that if dont get to all the questions due to this sales pitch that the offeror wont be able to allay any concerns the committee has in these areas & this may be harmful to the offerors technical standing & even whether its acceptable CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 448 What Would You Do? # 1 Situation: Offeror comes in with fancy slide show with graphics and sound & slick handouts Or, Could say the offeror can show as many slides as it can in the 10-15 minutes or so allotted to offerors introductions

However, any set-up time is part of the 10-15 minutes Can allow offeror to distribute the handouts May allow the 10-15 minutes to be from the handouts only without the slide show CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 449 What Would You Do? # 2 Offeror comes in with entourage of 10 people CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 450 What Would You Do? # 2 Situation: Offeror comes in with 10 people

This may or may not be a problem It depends on the value & complexity of the RFP & the offerors proposal If the procurement is very complex & the right people are brought this may be good Especially if offerors were told to bring many of these people because the committee wanted to hear & judge them CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 451 What Would You Do? # 2 Situation: Offeror comes in with 10 people

May be positive demonstration of the value the offeror attaches to winning the contract However, it also might be overkill The offeror might be trying to impress with numbers rather than the quality of its proposal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 452 What Would You Do? # 2 Situation: Offeror comes in with 10 people There may be a problem in managing the time for presentation Generally all those in attendance will want to speak Multiple responses will usually be given for all questions Have to be more diligent to keep the session moving so get through all questions or issues

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 453 What Would You Do? # 3 Offeror comes in with just 1 person CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 454 What Would You Do? # 3 Situation: Offeror only represented by 1 person This may or may not be a problem It depends on the value & complexity of the RFP & the offerors proposal For a relatively simple RFP & proposal this

could be fine if the right person is present It might simply be that not enough notice was given about the discussions session & other offeror personnel had previous commitments CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 455 What Would You Do? # 3 Situation: Offeror only represented by 1 person However, this may demonstrate a low level of interest in winning the award It also might show that the offeror doesnt have many personnel available to work on the contract It might prompt a question of how important the contract is to the offeror, especially if

other personnel were requested to be present CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 456 What Would You Do? # 4 Offeror starts off explaining how great it is & why it should receive the award & wont shut-up CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 457 What Would You Do? # 4 Situation: Offeror starts off explaining how great it is, why it should receive the award & wont shut-up

When the session was scheduled it should have been stated that the offeror only had 10-15 minutes for an initial presentation When this time is up need to cut off the offeror & start with the committees questions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 458 What Would You Do? # 4 Situation: Offeror starts off explaining how great it is & why it should receive the award & wont shut-up Need to emphasize that the committees questions are more important that this sales pitch so need to get on with the questions & get through them

State that if dont get to all the questions due to this sales pitch that the offeror wont be able to allay any concerns the committee has in these areas & this may be harmful to the offerors technical standing & even whether its acceptable CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 459 What Would You Do? # 4 Situation : Offeror starts off explaining how great it is & why it should receive the award & wont shut-up May be a deliberate attempt to kill time if the offeror doesnt expect to do well in hope that questions that arent asked wont be held against it May invite a question of whether this should be accepted by the committee as a demonstration of

the offerors inability to follow instructions or to focus CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 460 What Would You Do? # 5 Offeror gives only short, incomplete answers to almost all questions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 461 What Would You Do? # 5 Situation: Offeror gives only short, incomplete

answers to almost all questions Advise the offeror that the answers are not satisfactory Advise that the offerors technical standing, or even whether it is judged to be technically acceptable depends on full answers However, this may simply demonstrate that this offeror cant really do what is required CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 462 What Would You Do? # 6 Offeror gives lengthy, rambling answers to almost all questions, but never seems to really answer the questions CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 463 What Would You Do? # 6 Situation: Offeror gives lengthy, rambling answers to questions but doesnt really answer them Tell offeror that: It isnt answering the questions It must step up the pace to get through all the committees questions If this still doesnt help, begin to cut off the answers by saying that its apparent that the offeror cant or wont answer the question so the committee will move on to the next question CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 464 What Would You Do? # 7

Offeror gives answers which contradict the proposal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 465 What Would You Do? # 7 Situation: Offeror gives answers which contradict the proposal Tell the offeror that this is occurring If possible, specifically identify the section(s) of the proposal where the conflict is occurring CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 466

What Would You Do? # 7 Situation: Offeror gives answers which contradict the proposal Ask why this is occurring Are the offeror personnel at the discussion session unfamiliar with the proposal? If so, they will probably apologize for the error & start reading the proposal to see what they should say Are they seeking to change the proposal? Is the new information essentially clarifying in nature, not contradictory? CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 467 What Would You Do? # 7 Situation: Offeror gives answers which contradict the

proposal If the offerors personnel at the discussion session admit they are unfamiliar with the proposal Ask what else they might have stated incorrectly that you didnt pick up Ask for an explanation of why they have been sent to represent the offeror and explain the proposal when they dont know it State that this is not a good reflection on the offeror This situation is particularly troubling if the personnel who dont know the proposal are the ones who will be in positions of responsibility to implement it if that offeror should be selected for the award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

468 What Would You Do? # 7 Situation: Offeror gives answers which contradict the proposal If the offeror is seeking to change the proposal say this is not permissible at this time The proposal can only be changed in response to a request for a best and final offer (BAFO) State you will let all offerors know if and when a BAFO is being requested CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 469

What Would You Do? # 7 Problem: Offeror gives answers which contradict the proposal If the offeror is seeking to provide clarifying information Verify that you agree that the new information is essentially clarifying in nature, not new, contradictory information If you agree that the information is merely clarifying, if the clarification is substantive require the offeror to provide this information in writing CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 470 What Would You Do? # 8 Offeror comes in and seeks to

submit a revised technical or financial proposal CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 471 What Would You Do? # 8 Situation: Offeror comes in and seeks to submit a revised technical or financial proposal Do not take the revised proposal Say this is not permissible at this time The proposal can only be changed in response to a request for a best and final offer (BAFO) State you will let all offerors know if and when a BAFO is being requested

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 472 What Would You Do? # 9 Offeror personnel contradict each other in answering questions & squabble about which one is giving the right answer CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 473 What Would You Do? # 9 Situation: Offeror personnel contradict each other in answering questions & squabble about which one is giving the right answer

Tell the offeror that you observe this occurring State that you find this unsettling Ask why this is occurring Ask if this is the type of conduct that you should expect from this offeror if it would be selected for the award CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 474 What Would You Do? # 9 Situation: Offeror personnel contradict each other in answering questions & squabble about which one is giving the right answer

State that this conduct is taking time & may result in all questions from the committee not being asked or answered State that you would like one person from the offeror to assume the role of moderator and identify which offeror personnel is to answer each respective question from the committee CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 475 What Would You Do? # 9 Situation: Offeror personnel contradict each other in answering questions & squabble about which one is giving the right answer If the personnel identified by the moderator to answer a particular question is not the

person whom would seem appropriate (not the person who seems to have the authority for the action) ask for an explanation of this CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 476 What Would You Do? # 9 Situation: Offeror personnel contradict each other in answering questions & squabble about which one is giving the right answer State that because of the conflicting answers that the offeror should submit a written verification of the accuracy of what was said in response to all substantive questions I.e., the offeror should state which answer from its personnel is the official offeror answer CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 477 What Would You Do? # 10 Offeror personnel unsure about who should answer questions. Each sits there waiting for someone else to answer Multiple people are volunteered to give answers and none wants to do so CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 478 What Would You Do? # 10 Situation: Offeror personnel unsure about who

should answer questions Responses are the same as for What Would You Do issue # 9 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 479 What Would You Do? # 11 Offeror continually asks: Which other vendors have submitted proposals How many vendors have submitted proposals How its proposal compares to other offerors When told of areas in its proposal that are weak or need changed offeror asks what other offerors said for these areas

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 480 What Would You Do? # 11 Situation: Offeror seeks information about other offerors or their proposals Say that no information can be provided on: How many offerors submitted proposals or who they were The content of any other offerors proposal State that the offeror should have its own expertise and operating procedures & should not be looking to other offerors for answers on how to conduct its business or perform

the contract to be awarded CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 481 What Would You Do? # 12 When told of areas in its proposal that are weak or need changed offeror asks the evaluation committee what they would like to see (specific changes vendor should make) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 482 What Would You Do? # 12 Situation: Offeror seeks information about the preference of the committee

State that the committee will not provide answers for the offeror The committee identifies areas of uncertainty or concern but the offeror must decide how it will address or rectify these areas CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 483 Appendices CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 484 Evaluation Committee Guidelines

Appendix 1 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 485 1. All committee members (evaluators) must read all proposals. Members cannot divide-up the proposals with each member reading certain ones and reporting his/her impressions back to the others. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 486 2. Generally, all committee members must attend all committee meetings, including all discussion sessions with offerors. It is problematic for evaluators to base their evaluations on the reports or opinions of other evaluators rather than their firsthand

observations and analysis (this is also why all evaluators must read all proposals). Furthermore, not participating in a discussion session with offerors precludes an evaluator from asking his/her own questions of offerors or following-up on a given answer or statement by an offeror. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 487 2 (contd.) Members of an evaluation committee should have been selected because they add value to the committee through their unique experiences, skills, training, frame of reference, etc. They should not be considered so interchangeable that nothing is lost to the committee as a whole if one or more members dont participate in any significant activity of the committee. This participation requirement is so important that the procurement officer, possibly with consultation with agency legal staff, should consider the removal of any member that cannot/does not fully participate in the evaluation process or who does not do so conscientiously.

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 488 3. Committee members should fully discuss the contents of each proposal to ensure members read the same things. A frequent reason for differing opinions of an offeror or proposal is that one or more evaluators overlooked information in a proposal. Once there is agreement about what is in the proposals, there should be discussion to see if there are differing reactions to that information. If members have highly divergent evaluations of one or more proposals, the reason(s) for the divergence should be pinpointed and discussed. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 489

3 (contd.) If there is any doubt about what a proposal is saying, seek clarification. (Unless a proposal/offeror is so bad that the consensus is to eliminate the proposal as being not reasonably susceptible of being selected for award without clarifications or discussions). The evaluation should be based upon the evaluators reaction (judgement) to what the vendor is offering, not guessing about what is really being said or offered. In other words committee members shouldnt have different opinions about an offeror because they differ on what they think the offeror is saying. Members dont have to agree on how they react to a given piece of information, but they do need to agree on the information itself. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 490 4. Similarly, committee members need to be in agreement about the application of the evaluation criteria. They need to be uniform in what each criterion means and how to use it. All criterions are to be mutually exclusive. That is, any given

facet of an offerors proposal is to be evaluated against one, and only one criterion. If there is information in a proposal which doesnt obviously fall within the scope of an evaluation criterion, committee members, possibly with the guidance of the procurement officer, should discuss which criterion, if any, the topic most appropriately falls within. Once there is agreement on the most appropriate criterion, all members must evaluate the information under that criterion. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 491 4 (contd.) Conversely, if the consensus is that the topic/information does not fall within the scope of any evaluation criterion, it cannot be considered. The committee cannot agree at the evaluation stage to add an additional criterion or unreasonably expand the scope of an existing criterion. Also, the committee cannot agree to forego using a criterion that was listed in the RFP. CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 492 5. If the committee understands what a vendor is offering but doesnt like it, the vendor should be tactfully informed of the dislike during discussions. (Assuming the vendor isnt so bad that it needs to be eliminated before discussions are held). The vendor should then be allowed to revise its proposal. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 493 5 (contd.) Anything significant that a vendor will be downgraded on needs to be addressed to the offeror while it still has the opportunity to cure the apparent deficiency by answering questions about it and/or revising its proposal. A vendor should not have a major negative issue presented for the first time at a debriefing. This precept of addressing major deficiencies at the discussion phase usually results in less acrimonious debriefings and fewer

protests from unsuccessful offerors. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 494 6. Similarly, committee members need to be in agreement about the application of the evaluation criteria. They need to be uniform in what each criterion means and how to use it. All criterions are to be mutually exclusive. That is, any given facet of an offerors proposal is to be evaluated against one, and only one criterion. If there is information in a proposal which doesnt obviously fall within the scope of an evaluation criterion, committee members, possibly with the guidance of the procurement officer, should discuss which criterion, if any, the topic most appropriately falls within. Once there is agreement on the most appropriate criterion, all members must evaluate the information under that criterion. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 495

6. (contd.) Conversely, if the consensus is that the topic/information does not fall within the scope of any evaluation criterion, it cannot be considered. The committee cannot agree at the evaluation stage to add an additional criterion or unreasonably expand the scope of an existing criterion. Also, the committee cannot agree to forego using a criterion that was listed in the RFP. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 496 7. If the committee understands what a vendor is offering but doesnt like it, the vendor should be tactfully informed of the dislike during discussions. (Assuming the vendor isnt so bad that it needs to be eliminated before discussions are held). The vendor should then be allowed to revise its proposal. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

497 7. (contd.) Anything significant that a vendor will be downgraded on needs to be addressed to the offeror while it still has the opportunity to cure the apparent deficiency by answering questions about it and/or revising its proposal. A vendor should not have a major negative issue presented for the first time at a debriefing. This precept of addressing major deficiencies at the discussion phase usually results in less acrimonious debriefings and fewer protests from unsuccessful offerors. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 498 8. An informal objective of discussions should be to elicit compliments from offerors on the thoroughness of the evaluation committees review of their proposals and the perceived fairness of the proceedings. Perception is important. An unsuccessful offeror that believes it has been treated fairly and competently evaluated is far less likely to be disgruntled, including less likely

to file a protest. It is also more likely to continue bidding on State procurements. It doesnt benefit the State for competition to dry up because vendors feel they have been mistreated. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 499 9. Offerors must be given both a reasonable amount of time and the same amount of time to respond to questions, revise proposals, provide additional information, etc. This includes non-business days as well as business days. For example, an offeror that meets with the evaluation committee on a Tuesday and is given 3 business days to respond to provide additional or revised material would have until the end of the day on Friday to do so. In this case the offeror has both 3 business days and 3 calendar days to respond. (Assuming no holiday occurs in these 3 days). However, an offeror meeting on Wednesday and having 3 business days to respond has until the end of the day on the following Monday. (Again assuming no holiday). While this is 3 business days, it is 5 calendar days. Hence, if the

second vendor chose to work over a weekend it would have 2 extra days to prepare its response. This is unfair to the offeror which met on Tuesday. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 500 9 (contd.) In this situation, each offeror should be given either 2 business days to respond (if this would be sufficient time) or 4 business days. Under the 2 business day scenario the responses would be due on Thursday for the Tuesday meeting and Friday for the Wednesday meeting, which is also 2 calendar days for each offeror. (Assuming no holiday). For the 4 business day scenario the responses would be due on Monday for the Tuesday meeting and Tuesday for the Wednesday meeting. (Assuming no holiday). Each offeror would thus have 6 calendar days to respond. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

501 10. The objective of the competitive sealed proposals procurement method is to obtain the best deal for the State. Accordingly, it should be assumed that, if given a chance, all offerors, even the offeror with the best initial proposal, can and will improve their proposals during the discussion process. Hence, earnest discussions should occur with all offerors that the committee believes have a reasonable chance of producing an acceptable proposal following discussions and revision. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 502 10 (contd.) In other words the initial judgement of an offerors susceptibility for award should be based partially on the offerors original submission and partially on its perceived capability of being deemed acceptable after discussions and revision. In particular, otherwise acceptable or potentially acceptable offerors that have included unacceptable terms or conditions should generally

be permitted to withdraw the offending language before being rejected outright. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 503 11. The objective is to keep vendors in the running, not eliminate them as quickly as possible. Care should be taken so that competition is not unduly restricted by over zealous elimination of offerors at the technical proposal evaluation phase. Evaluators and the procurement officer should be mindful that the top technically ranked offerors may end up being unaffordable. (Remember that the financial proposals should not yet have been opened while the initial technical proposal discussions are taking place). This is one of the principal reasons why meaningful discussions should be conducted with every potentially acceptable offeror. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

504 11. (contd.) If the top ranked offerors are unaffordable and all other offerors have been eliminated, no award can occur. The agency either does without the service the RFP was intended to provide or has to start over and re-solicit. Even if the top ranked offeror is affordable, it will never be known how much money might have been saved if earnest discussions had resulted in an offeror whose initial proposal was marginal, or even unacceptable, being able to revise their proposal or satisfactorily address questions from the committee so that they vault into the acceptable range. Moreover, earnest discussions might similarly help resident and/or minority businesses. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 505 12. The key factor in determining which offeror is finally selected as being the most advantageous offeror is worth. The general procedure is to begin with the offeror that is the highest ranked

technical offeror and, assuming that offeror is not also the lowest bidder, ask the simple question of whether that offeror is worth the cost differential over any and all other offerors. If the collective answer of the evaluation committee is yes, this highest ranked technical offeror is worth its higher price versus any other offeror, the committee then must put into writing exactly why they believe this is the case. The procurement officer must then uphold the cogent factors supporting this most advantageous offeror determination. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 506 12. (contd.) Alternatively, if the committee and/or the procurement officer do not feel the highest ranked technical offeror is worth its price differential, the same process is followed for the next highest technically ranked offeror. This process continues until the committee and the procurement officer staunchly agrees that a particular offeror is worth its cost differential over all other offerors.

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 507 DBM Solicitation Evaluation Criteria Appendix 2 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 508 DBM Solicitation Evaluation Criteria & Form The following 11 slides are excerpts from an evaluation form that was used by DBM for a number of years

While this form has partially become obsolete with the creation of the various solicitation templates, many of the listed elements are still pertinent CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 509 DBM Solicitation Evaluation Form Criteria #1: Method The following must be properly identified and consistently applied throughout the RFP or IFB: q Procurement Method q Type of Contract, and q Award Status (single or multiple awards) Multiple awards are to specify the number of awards or circumstances for making award selections, and how to determine which contractor will be used for any given occasion of need. This includes the issuance of master contract awards with competition for task

orders. 510 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work DBM Solicitation Evaluation Form Criteria #2: Scope Focuses on the objective (performance), not the strategy (design). The Scope must be concisely and clearly written with: q Correct grammar, punctuation, spelling, syntax, and esthetics, q No obviously ambiguous language, and q Technical jargon and abbreviations held to a minimum and clearly defined when used. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 511 Criteria # 2 Scope (Contd)

The Scope also must clearly provide: q Sufficient information for vendors to successfully bid or prepare proposals, q If Minimum Qualifications are used that they are: q not excessive (such that they would unreasonably exclude vendors) q essential to assure adequate performance q properly distinguished from evaluation factors, q Deliverables, CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 512 Criteria # 2 Scope (Contd)

q Timeframes, q Performance Standards and who determines whether they are met, q State obligations (for example staff, space, data), q Monitoring mechanisms/processes, q Clearly defined Activities and Responsibilities, i.e. what is to be done, who is responsible, when things occur, and if appropriate, how things will be

done, q When applicable, multiple and alternate proposal submission language, q Appropriately used and calculated liquidated damages, and q When Payments will be made (defined in terms of timeframe and deliverables). 513 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work DBM Solicitation Evaluation Form Criteria #3: Required Response The solicitation clearly defines what is expected of the bidder/offeror: q

In the proposal or bid, and q This is appropriately distinguished from what the contractor is to do after the contract is awarded. (Deliverables under #2, Scope) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 514 DBM Solicitation Evaluation Form Evaluation Criteria # 4 Technical Criteria q Are clear and appropriate for both

the contract requirements and the requested content of proposals, q Permit proper assessment of the differences among offerors, q Clearly state how the winning vendor will be selected, and q Correctly use reciprocal preference and economic benefit considerations. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 515 DBM Solicitation Evaluation Form Criteria # 5 Pricing Method q Mirrors deliverables, and billing

requirements (matches billing schedule and cost breakdown), q Is logical and appropriately weighted with clear instructions for its completion, and q Results in a single bid price (actual fixed price or price for a model of assumed circumstances) which is comparable among bidders/offerors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 516 DBM Solicitation Evaluation Form Criteria # 6 General The following information/attachments must be included: q Procurement Officer and Contract Monitor are identified and appropriate

q Mandatory & other essential clauses, q Forms/affidavits, q The proposed contract with proposed payment terms, warranty information, etc., q Other attachments referenced in CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 517 Criteria # 6: General (Contd) The following information/attachments must be included:

q Necessary instructions such as pre-bid conference, how questions will be answered, escalation clauses, q Term of contract (starting and ending dates, provision for options), q MBE requirements, if applicable q Non-visual access clause when appropriate, q Hiring agreement clause when appropriate, & q Appropriate types and dollar values of insurance and bonds CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 518 Criteria # 6: General (Contd) The overall solicitation document must be logically organized (have good flow), and must not contain:

q Contradictions between different parts of the solicitation, q Excessive redundancy (for the most part, Say it once; Say it well; Dont say it again), and q Inconsistent and inappropriately used terminology. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 519 DBM Solicitation Evaluation Form Criteria # 7 Adequate Time Allowance Adequate Time Allowance is provided to conduct all aspects of the

procurement including: q The pre-bid conference, q Vendor questions and responses, q Proposal review with discussions, q Required State approvals, and q Start-up time CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 520 Appendix 3 Historical information on when Competitive Sealed Proposals formerly could be used CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 521 When CSP Can Be Used CSP is the preferred procurement method for the procurement of human, social, cultural, or educational services or real property leases As the preferred procurement method: No determination or special approval is needed to use CSP for procurements for any of these 4 types of services or for real property leases But sealed bidding can still be done for other than real property leasing if deemed appropriate

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 522 When CSP Can Be Used Each of human, social, cultural, and educational services is defined in COMAR Title 21 (see the following 4 slides) All but cultural services require the direct provision of services to third-party clients Third-party clients is also defined in COMAR Title 21 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

523 Cultural Services Means services that are provided directly to third-party clients or to the public under a contract the primary purpose of which is the direct provision of cultural services. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 524 Educational Services Means services procured by the Departments of Health & Mental Hygiene, Human Services, Labor, Licensing, and Regulation, or Juvenile Services, the Office for Individuals with Disabilities, or the Dept.

of Aging in order to provide training directly to third-party clients under a contract the primary purpose of which is the direct provision of educational services. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 525 Human Services Means services procured by the Departments of Health & Mental Hygiene, Human Services, Labor, Licensing, and Regulation, Juvenile Services, or the Dept. of Aging in order to provide support, care, or shelter directly to third-party clients under a contract the primary purpose of which is the direct provision of these services. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

526 Social Services Means services procured by the Departments of Health & Mental Hygiene, Human Services, Labor, Licensing, and Regulation, Juvenile Services, the Office for Individuals with Disabilities, or the Dept. of Aging in order to provide support, care, or shelter directly to third-party clients under a contract the primary purpose of which is the direct provision of social services. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 527 Third-Party Clients Means persons who are receiving

human, social, cultural, or educational services primarily because they are aged, indigent, disadvantaged, unemployed, mentally or physically ill, handicapped, displaced, or minors CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 528 When CSP Can Be Used Only 5 agencies can procure human services DHMH DHS DLLR DJS DOA CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 529 When CSP Can Be Used These same 5 agencies can procure educational or social services A 6th agency can also use CSP for educational or social services (but not for human services) The Office for Individuals with Disabilities Any state agency may procure cultural services Cultural services can be provided to third-party clients or to the public

CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 530 When CSP Can Be Used DGS is the primary agency that can enter into real property leases Other agencies subject to Title 21 with real property leasing authority are the: Port Administration Aviation Administration CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 531

Appendix 4 COMAR RFP Requirements (21.05.03.02 A and 03 A) CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 532 COMAR Requirements for RFPs (Per 21.05.03.02 A and 21.05.03.03 A) 21.05.03.02 A: The request for proposals shall include: 1. 2. 3. Information concerning proposal submission

requirements, including the date, time and place for the receipt of proposals. The evaluation factors and an indication of the relative importance of each evaluation factor, including price. A work statement or scope of services statement, performance schedule, and any special instructions CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 533 The request for proposals shall include: 4. A statement that discussions may be conducted with those responsible offerors that submitted proposals initially judged by the procurement officer to be reasonably susceptible of being selected for award. However, if the request for proposals so notifies all offerors, discussion need not be conducted: a. With respect to prices that are fixed by law or regulation, except that consideration shall be given to competitive terms

and conditions b. If time of delivery or performance does not permit discussions; or c. If it can be demonstrated clearly from the existence of adequate competition or accurate prior cost experience with the particular item, that acceptance of an initial offer without negotiation would result in a fair and reasonable price. 534 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work The request for proposals shall include: 5. 6. 7. 8. A statement of when and how price proposals should be submitted. A statement that price proposals are irrevocable for 90 days following the closing date for submission of

price proposals or best and final offers, if requested. However, a procurement officer may reduce or increase the period of time if he determines it to be in the States best interest provided that the period is reasonable and is clearly stated in the request for proposals. A statement whether multiple or alternate proposals may be submitted and their treatment The mandatory solicitation requirements set forth in COMAR 21.05.08. 535 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work The request for proposals shall include: All required terms and conditions. 10. A requirement that the receipt of any amendments to the request for proposals be acknowledged by offerors known by the procurement officer to have been issued, or otherwise to have received, the request for proposals. 9.

536 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work COMAR 21.05.03.03 A: Evaluation 1. 2. 3. The evaluation shall be based on the evaluation factors set forth in the request for proposals and developed from both the work statement and price. Technical proposals and price proposals shall be evaluated independently of each other. Economic Benefit Evaluation Factor (The narrative of this section essentially appears as the wording of Section 5.4.2.15 of the standard RFP template.) CSP - 4/2018

Copyright 2018 Md. Dept. of Health - Unpublished Work 537 COMAR 21.05.03.03 A: Evaluation 4. 5. 6. Numerical rating systems may be used but are not required. Factors not specified in the request for proposals may not be considered. Initial evaluations may be conducted and recommendation for award made by an evaluation committee. Final evaluations, including evaluation of the recommendation of the evaluation committee, if any, shall be performed by the procurement officer and the agency head or designee. CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

538 Appendix 5 Fast and Good MORE ON THE 2001 STATE ELECTIONS SYSTEM PROCUREMENT CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work 539 Example of Fast & Good A new statewide elections system (voting machines) for the 2002 elections The need for a solicitation was learned on 5/11/2001 The RFP was advertised in the Contract Weekly 7/17/2001 Total elapsed time 67 days (roughly 2 months)

540 Due to other commitments work wasnt started on 5/11 Most of the RFP was written in about 30 days from midJune to 7/17/2001 For a 3 week period the core involved personnel did virtually nothing else & put in lots of compensatory time CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work Example of Fast & Good Complication On 7/16/2001 was told not to release the RFP On 7/18/2001 was told to reduce the system from statewide to just 4 counties

A revised RFP was issued on 7/20/2001 541 Only those with the most antiquated equipment 47 hours from the time direction was received to reduce the RFP to 4 counties CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work State Elections System Procurement This was one of first procurements of this type in the country (for electronic voting machines) The awarded contract would exceed $16 million

after options were exercised, just for the 4 counties 542 There were only 3 known predecessors, all for counties, not states And would have exceeded $50 million if it would have stayed a statewide procurement Shortly thereafter the State of Georgia used the RFP as the basis for its statewide election system procurement CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work State Elections System Procurement (Cont'd)

The State Board of Elections didnt have the resources to do this procurement SBE has about 35 employees Asked DBM for help The procurement was approached as a project using a team approach Regular & frequent status meetings were held Timeframes were set Needed resources were identified Resources were made available 543 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work Resources Were Made

Available From DBM: Director of Procurement Policy and Administration Procurement Chief Assistant Procurement Chief Who was the procurement officer Assistant Attorneys General IT personnel Budget personnel CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

544 Resources Were Made Available (Cont'd) From SBE: Executive Director 2 policy staff Assistant Attorney General Outside technical advisor SBE Board members 545 CSP - 4/2018 Support and

decisions from DBM Secretary Secretary of State (SBE was under the Secretary of State then) Governors Chief of Copyright 2018 Md. Dept. of Health - Unpublished Work Staff What Does This Prove? With the right resources, a procurement doesnt have to take 6 to 12 months or more Specifications for a never previously done $17 million procurement, were written in about 30 days

Nothing in the law, regulations or control agency policy or procedures prohibits the relatively rapid development of a high quality solicitation document Drafting time can be cut significantly with: The use of the proper process A reasonable level of effort from skilled, knowledgeable people 546 CSP - 4/2018 Copyright 2018 Md. Dept. of Health - Unpublished Work

Recently Viewed Presentations

  • Neck Region - Misericordia University

    Neck Region - Misericordia University

    Be able to describe the fascia of the neck and apply its significance to clinically important conditions. Be able to describe the borders of triangles of the neck and list their components. Be able to identify thyroid structures including anatomical...
  • Council of the European Union General Secretariat Directorate-General

    Council of the European Union General Secretariat Directorate-General

    Director. Petr BLÍŽKOVSKÝ. Post-2020 CAP reform. Greener . policy for modern and diversified . farming. The opinions expressed in this presentation are those of the author and do not necessarily reflect the official policy of the European Union.
  • Mechanisms and Management of an Incomplete Epidural Block for ...

    Mechanisms and Management of an Incomplete Epidural Block for ...

    Preventing Epidural Block Failure. Saline for LOR (less unblocked segments) Multi-holed catheter. Catheter 2-4 cm in epidural space (prevents unilateral spread) Using epidural adjuncts has not been proven to decrease block failure but opioids decrease the amount of LA needed...
  • Exercise - Getting Students Involved

    Exercise - Getting Students Involved

    Aims of the workshop To understand and experience Enquiry-Based Learning (EBL) To understand that EBL is 'nothing new' To discuss some practical concerns about applying EBL methods in your own small group teaching [Not about designing EBL exercises] Approach The...
  • EPSAP Professional Engineer Submittal Instructions

    EPSAP Professional Engineer Submittal Instructions

    Step #3 Choose the option that verify the Final Signature File Authentication Codes matches the ones on your PE Signature Document then click Continue. Note: the PE Signature document can be printed from EPSAP Main Menu. As well as the...
  • Diapositive 1

    Diapositive 1

    The detection instrument for the quantitative determinationvaryfrom a single Petridish to different dilutions and MPN system of diverse complexity. Intrinsicvariablity of microbiologicaldeterminations ... Every technical step of the method adds to the total variability of the measurement: Sub ...
  • Chemistry 2.2 Identify Ions AS 91162 Ionic Equations

    Chemistry 2.2 Identify Ions AS 91162 Ionic Equations

    Chemistry 2.2 Identify Ions AS 91162 Ionic Equations Cations ion Precipitation equation precipitate Complex ion equation NH4+ No equation No ppt No equation
  • THE EARLIEST HEROES PROMETHEUS AND IO (BACKGROUND) Prometheus-

    THE EARLIEST HEROES PROMETHEUS AND IO (BACKGROUND) Prometheus-

    Zeus, one not know for his faithfulness, sees Europa picking flowers and immediately is attracted to her. In order to come in contact with Europa, Zeus transformed himself into a white bull. He then walked over to her slowly and...