Executive Summary
1.Introduction
Subsequent to the actual initiation of Number Portability (NP) services,
Service Providers must deploy certain systems and databases (SOA/LSMS)
in order for the critical transaction flows to be exchanged between the
respective participating Service Providers and the Number Portability
Administration Center's SOA and master SMS database (NPAC/SMS). To
insure that the Service Providers SOA/LSMS conform to certain industry
standards and are technically and operationally configured correctly a
significant effort has been devoted to the development of a well-defined
NPAC Internal Verification Test Plan, Inter-operability Test Plan, and a
NPAC/SMS Turn Up Test Plan that clearly outlines the Service Providers
responsibilities and test actions. The purpose of these test plans,
therefore, is to document the testing requirements, methodology and time
lines for NPAC SMS/SOA testing, individual Service Provider SOA/LSMS
level tests and the interactive Service Provider testing with the
NPAC/SMS which are all necessary for the successful and timely
implementation of number portability
in Illinois.
1.1 Testing Scope
As transactions go both ways between the NPAC/SMS and the LSMS/SOA, it
is an explicit requirement that the LSMS/SOA respond correctly to the
NPAC/SMS initiated tests, and that the SOA or LSMS initiated tests be
implemented according to the test plan and in accordance with industry
accepted data exchange protocols. Therefore the scope of the NPAC/SMS
Internal Verification Test Plan and Turn Up Test Plan as well as the
NPAC SMS Inter-operability Test Plan is to provide service support
documents which will enable participating Service Providers and their
respective SOA and LSMS vendor(s) to readily identify their specific
inter-operability testing responsibilities, understand the tests that are
necessary to be performed, and how to interpret the results of these
tests for follow-on regression testing,
if necessary.
The immediate goal of these initial test efforts is to meet the July 1,
1997, FCC test date for "live" customer testing. This is to be
accomplished by initiating Turn Up Testing with Service Providers that
are test ready beginning as early as March 1997. It should also be
noted that Service Providers may also wish to complete certain Service
Provider to Service Provider tests,
if applicable.
In view of the dynamic nature of Number Portability, it is envisioned
that the testing processes will continue as a continuing on-going
activity and mission critical function of the NPAC. It is currently
accepted that as each new Service Provider enter the local market their
LSMS/SOA will need to enter into Inter-operability Certification with the
Lockheed Martin NPAC/SMS Test Lab, and NPAC Turn Up Testing, as a pre-
requisite to actual service initiation. Also to be considered is the
need for internal NPAC regression testing and Inter-operability Testing
with the Service Providers for each new software release of the NPAC's
SOA, or SMS.
1.2 Service Provider: SOA/LSMS Preparation for Inter-operability
Testing and Certification
Each participating Service Provider will be required to design, code and
internally test their LSMS/SOA systems prior to initiating
interconnected testing with the NPAC/SMS. Since the LSMS has a peer-to-
peer relationship with the NPAC SMS, the LSMS will perform the role of
both Manager and Agent from a CMIP perspective and requires that the
scripted LSMS/SOA initiated test cases and drivers (defined in the
Lockheed Martin NPAC/SMS Inter-operability Test Plan) must be implemented
and executable by the Service Provider before actual testing with the
Lockheed Test Center can be completed. It is also a requirement that
the Service Provider's OSI/RFC1006 stack is CTS-3 certified before
actual interactive testing process with the Test Center can begin. Once
CTS-3 certification is obtained, the Service Provider will be able to
schedule test times with the Lockheed-Martin
IMS Test Lab.
1.3 Service Provider: SOA/LSMS Inter-operability Testing and
Certification
with the LMIMS Test Lab
Inter-operability testing between the Service Provider's LSMS/SOA and the
NPAC/SMS test lab is designed as a series of test phases which have been
developed to ensure that each Service Provider meets the technical and
operational processing requirements necessary for the transactional
exchange of ported customer information. Each Service Provider will be
provided test access in an isolated environment that is intended to
allow for careful and methodical execution of each test phase with
evaluation times incorporated in the testing schedule in case system or
software modifications are required. These test phases are: (1)
Connectivity, (2) Managed Objects, Security, and Recovery, (3)
Application to Application Testing. Total elapsed time to complete all
test phases is estimated to take approximately six weeks for an LSMS
interface and three weeks for a SOA
interface.
The Service Provider will be responsible for scheduling test times with
Lockheed-Martin. It should be noted that the Service Provider (or
through its designated agent) may test the SOA and LSMS separately,
however, the Service Provider is required to have completely implemented
all SOA/LSMS initiated test cases and drivers before actual testing can
be completed. As noted previously, the Service Provider must also be
able to furnish documentation to the Test Center that their OSI/RFC1006
stack has been CTS-3 certified.
Inter-operability Test scenarios contain representative samples of
mandatory and optional tests that are designed to address general areas
of LSMS/SOA conformance to the ISO/ITU standards for OSI Conformance
Testing Methodology and Framework. Briefly, LSMS/SOA conformance is to
be validated through Basic Interconnection Tests (S2S and limited
Security tests), Capability Tests (MOC), and Behavior Tests (Security,
MOC, and Recovery). As noted previously, there are five specific test
components contained in the conformance tests which must be completed
between the Service Provider and the NPAC/SMS Test Lab in order to
obtain Certification. The mandatory
tests are further described below.
* Stack-to-Stack (S2S).Stack-to-Stack, or as it is referred to in the
Inter-operability Test Plan as "S2S," tests the ACSE/ROSE connectivity
between the LSMS/SOA of the Service
Provider with the NPAC SMS.
* Managed Object Conformance (MOC).Each Managed Object (MO) in the
system, as described in the GDMO file, will be subjected to tests that
address their basic operations, with the exception for the application-
to-application level of behavior of
the MOs.
* Security. The purpose of this test sequence is to ascertain the
functionality of each item in the Inp Access Control field and is designed
to cover system Id, cmip Departure Time, sequence Number, list Id, key Id,
and signature sub-fields. Testing of the signature test case may be
postponed until the end of the test process, however, the other Security
test cases must be completed successfully
prior to MOC testing.
* Recovery. The focus of this test group is to ensure recovery in the
event of a "crash" of either the Service Providers LSMS/SOA or the
NPAC/SMS. In addition, this test group will also be used to determine
if the SOA or LSMS can switch to the "backup" NPAC/SMS, when instructed
to do so by the serving NPAC/SMS.
After completion of NPAC/SMS inter-operability testing, a Test Report
will be created by the Test Center, in the standardized format outlined
in the Inter-operability Test Plan, and a copy forwarded to Lockheed
Martin IMS and a copy to the Service Provider. The Test Lab will
maintain copies of the test results along with a matrix of which Service
Providers have completed which phases of the Inter-operability Test.
Upon successful execution of all mandatory test cases, a certification
shall be issued to the Service Provider authorizing connection to the
production NPAC/SMS.
1.4 Service Provider/NPAC SMS Turn-up
Testing
Once a Service Provider has obtained LSMS and/or SOA certification,
then connection to the production NPAC/SMS will be permitted so that
Turn-up Testing can be conducted.
Although the Turn-up Test Plan is based upon elements similar to the
Inter-operability Test Plan, and the NPAC Internal Verification Test
Plan, it addresses issues specific to activation and testing of the
Service Provider systems within the
real production environment.
The scope of this testing is limited to conducting a subset of test
cases identified in the Inter-operability Test Plant and the NPAC
internal Verification Test Plan, and the addition of certain test cases
pertinent to the environments such as back up and recovery. These test
cases will be conducted in the production environment with each Service
Provider, allowing the Service Provider to test and repair any problems
with LSMS or SOA functionality. Through configurable NPAC NPA-NXX
screening by each individual Service Provider, the testing environment
can provide isolation for simultaneous
Service Provider Testing.
1.5NPAC Internal Verification Testing
Process:
Key to the critical path leading to Inter-operability Testing between the
Service Providers LSMS/SOA and the NPAC/SMS/SOA is the need for
completing internal testing of the NPAC and associated service
components, i.e.; the master SMS and SOA interfaces. This testing
process is referred to as the NPAC Internal
Verification Test Plan.
Rigorous internal testing of the NPAC will validate the existence and
system functionality of its specific service and performance
capabilities. The NPAC Internal Verification Test will begin with
comprehensive unit tests and will continue through individual testing of
the software module, integration testing of all system elements,
simulated user testing and conclude with LMIMS internal acceptance
tests. Of specific interest during the test phases is the systems
behavior and compliance, as defined by the NPAC SMS Interoperable
Interface Specifications, to correctly handle various Agent/Manager
protocol transactions associated with notifications and replies
necessary for LSMS/SOA communications.
Internal NPAC testing will be comprehensive and thorough and will cover
a full range of dynamic system and service requirements, however, since
the number of combinations of possible events (existing and future) is
astronomical, such testing will, by necessity, be focused on currently
identified service requirements and performance criteria. Tests cases
will cover all functional components of the NPAC and will include areas
such as Network Communications, Internal System Audits, Service Provider
interaction, Subscription Version, System Recovery, Security and Access
Control. Upon completion of the NPAC Internal Verification Test, a test
report will be created which will evaluate the system's behavior
experienced during the testing phases and certify that all system
components function according to original design specification documents
and functionally comply, as negotiated, with Service Provider
inter-operability requirements.
NPAC Testing Summary January 03, 1997
-Draft-