Executive Summary

1.Introduction

Subsequent to the actual initiation of Number Portability (NP) services,

Service Providers must deploy certain systems and databases (SOA/LSMS)

in order for the critical transaction flows to be exchanged between the

respective participating Service Providers and the Number Portability

Administration Center's SOA and master SMS database (NPAC/SMS). To

insure that the Service Providers SOA/LSMS conform to certain industry

standards and are technically and operationally configured correctly a

significant effort has been devoted to the development of a well-defined

NPAC Internal Verification Test Plan, Inter-operability Test Plan, and a

NPAC/SMS Turn Up Test Plan that clearly outlines the Service Providers

responsibilities and test actions. The purpose of these test plans,

therefore, is to document the testing requirements, methodology and time

lines for NPAC SMS/SOA testing, individual Service Provider SOA/LSMS

level tests and the interactive Service Provider testing with the

NPAC/SMS which are all necessary for the successful and timely

implementation of number portability in Illinois.

1.1 Testing Scope

As transactions go both ways between the NPAC/SMS and the LSMS/SOA, it

is an explicit requirement that the LSMS/SOA respond correctly to the

NPAC/SMS initiated tests, and that the SOA or LSMS initiated tests be

implemented according to the test plan and in accordance with industry

accepted data exchange protocols. Therefore the scope of the NPAC/SMS

Internal Verification Test Plan and Turn Up Test Plan as well as the

NPAC SMS Inter-operability Test Plan is to provide service support

documents which will enable participating Service Providers and their

respective SOA and LSMS vendor(s) to readily identify their specific

inter-operability testing responsibilities, understand the tests that are

necessary to be performed, and how to interpret the results of these

tests for follow-on regression testing, if necessary.

The immediate goal of these initial test efforts is to meet the July 1,

1997, FCC test date for "live" customer testing. This is to be

accomplished by initiating Turn Up Testing with Service Providers that

are test ready beginning as early as March 1997. It should also be

noted that Service Providers may also wish to complete certain Service

Provider to Service Provider tests, if applicable.

In view of the dynamic nature of Number Portability, it is envisioned

that the testing processes will continue as a continuing on-going

activity and mission critical function of the NPAC. It is currently

accepted that as each new Service Provider enter the local market their

LSMS/SOA will need to enter into Inter-operability Certification with the

Lockheed Martin NPAC/SMS Test Lab, and NPAC Turn Up Testing, as a pre-

requisite to actual service initiation. Also to be considered is the

need for internal NPAC regression testing and Inter-operability Testing

with the Service Providers for each new software release of the NPAC's

SOA, or SMS.

1.2 Service Provider: SOA/LSMS Preparation for Inter-operability

Testing and Certification

Each participating Service Provider will be required to design, code and

internally test their LSMS/SOA systems prior to initiating

interconnected testing with the NPAC/SMS. Since the LSMS has a peer-to-

peer relationship with the NPAC SMS, the LSMS will perform the role of

both Manager and Agent from a CMIP perspective and requires that the

scripted LSMS/SOA initiated test cases and drivers (defined in the

Lockheed Martin NPAC/SMS Inter-operability Test Plan) must be implemented

and executable by the Service Provider before actual testing with the

Lockheed Test Center can be completed. It is also a requirement that

the Service Provider's OSI/RFC1006 stack is CTS-3 certified before

actual interactive testing process with the Test Center can begin. Once

CTS-3 certification is obtained, the Service Provider will be able to

schedule test times with the Lockheed-Martin IMS Test Lab.

1.3 Service Provider: SOA/LSMS Inter-operability Testing and

Certification

with the LMIMS Test Lab

Inter-operability testing between the Service Provider's LSMS/SOA and the

NPAC/SMS test lab is designed as a series of test phases which have been

developed to ensure that each Service Provider meets the technical and

operational processing requirements necessary for the transactional

exchange of ported customer information. Each Service Provider will be

provided test access in an isolated environment that is intended to

allow for careful and methodical execution of each test phase with

evaluation times incorporated in the testing schedule in case system or

software modifications are required. These test phases are: (1)

Connectivity, (2) Managed Objects, Security, and Recovery, (3)

Application to Application Testing. Total elapsed time to complete all

test phases is estimated to take approximately six weeks for an LSMS

interface and three weeks for a SOA interface.

The Service Provider will be responsible for scheduling test times with

Lockheed-Martin. It should be noted that the Service Provider (or

through its designated agent) may test the SOA and LSMS separately,

however, the Service Provider is required to have completely implemented

all SOA/LSMS initiated test cases and drivers before actual testing can

be completed. As noted previously, the Service Provider must also be

able to furnish documentation to the Test Center that their OSI/RFC1006

stack has been CTS-3 certified.

Inter-operability Test scenarios contain representative samples of

mandatory and optional tests that are designed to address general areas

of LSMS/SOA conformance to the ISO/ITU standards for OSI Conformance

Testing Methodology and Framework. Briefly, LSMS/SOA conformance is to

be validated through Basic Interconnection Tests (S2S and limited

Security tests), Capability Tests (MOC), and Behavior Tests (Security,

MOC, and Recovery). As noted previously, there are five specific test

components contained in the conformance tests which must be completed

between the Service Provider and the NPAC/SMS Test Lab in order to

obtain Certification. The mandatory tests are further described below.

* Stack-to-Stack (S2S).Stack-to-Stack, or as it is referred to in the

Inter-operability Test Plan as "S2S," tests the ACSE/ROSE connectivity

between the LSMS/SOA of the Service Provider with the NPAC SMS.

* Managed Object Conformance (MOC).Each Managed Object (MO) in the

system, as described in the GDMO file, will be subjected to tests that

address their basic operations, with the exception for the application-

to-application level of behavior of the MOs.

* Security. The purpose of this test sequence is to ascertain the

functionality of each item in the Inp Access Control field and is designed

to cover system Id, cmip Departure Time, sequence Number, list Id, key Id,

and signature sub-fields. Testing of the signature test case may be

postponed until the end of the test process, however, the other Security

test cases must be completed successfully prior to MOC testing.

* Recovery. The focus of this test group is to ensure recovery in the

event of a "crash" of either the Service Providers LSMS/SOA or the

NPAC/SMS. In addition, this test group will also be used to determine

if the SOA or LSMS can switch to the "backup" NPAC/SMS, when instructed

to do so by the serving NPAC/SMS.

After completion of NPAC/SMS inter-operability testing, a Test Report

will be created by the Test Center, in the standardized format outlined

in the Inter-operability Test Plan, and a copy forwarded to Lockheed

Martin IMS and a copy to the Service Provider. The Test Lab will

maintain copies of the test results along with a matrix of which Service

Providers have completed which phases of the Inter-operability Test.

Upon successful execution of all mandatory test cases, a certification

shall be issued to the Service Provider authorizing connection to the

production NPAC/SMS.

1.4 Service Provider/NPAC SMS Turn-up Testing

Once a Service Provider has obtained LSMS and/or SOA certification,

then connection to the production NPAC/SMS will be permitted so that

Turn-up Testing can be conducted.

Although the Turn-up Test Plan is based upon elements similar to the

Inter-operability Test Plan, and the NPAC Internal Verification Test

Plan, it addresses issues specific to activation and testing of the

Service Provider systems within the real production environment.

The scope of this testing is limited to conducting a subset of test

cases identified in the Inter-operability Test Plant and the NPAC

internal Verification Test Plan, and the addition of certain test cases

pertinent to the environments such as back up and recovery. These test

cases will be conducted in the production environment with each Service

Provider, allowing the Service Provider to test and repair any problems

with LSMS or SOA functionality. Through configurable NPAC NPA-NXX

screening by each individual Service Provider, the testing environment

can provide isolation for simultaneous Service Provider Testing.

1.5NPAC Internal Verification Testing Process:

Key to the critical path leading to Inter-operability Testing between the

Service Providers LSMS/SOA and the NPAC/SMS/SOA is the need for

completing internal testing of the NPAC and associated service

components, i.e.; the master SMS and SOA interfaces. This testing

process is referred to as the NPAC Internal Verification Test Plan.

Rigorous internal testing of the NPAC will validate the existence and

system functionality of its specific service and performance

capabilities. The NPAC Internal Verification Test will begin with

comprehensive unit tests and will continue through individual testing of

the software module, integration testing of all system elements,

simulated user testing and conclude with LMIMS internal acceptance

tests. Of specific interest during the test phases is the systems

behavior and compliance, as defined by the NPAC SMS Interoperable

Interface Specifications, to correctly handle various Agent/Manager

protocol transactions associated with notifications and replies

necessary for LSMS/SOA communications.

Internal NPAC testing will be comprehensive and thorough and will cover

a full range of dynamic system and service requirements, however, since

the number of combinations of possible events (existing and future) is

astronomical, such testing will, by necessity, be focused on currently

identified service requirements and performance criteria. Tests cases

will cover all functional components of the NPAC and will include areas

such as Network Communications, Internal System Audits, Service Provider

interaction, Subscription Version, System Recovery, Security and Access

Control. Upon completion of the NPAC Internal Verification Test, a test

report will be created which will evaluate the system's behavior

experienced during the testing phases and certify that all system

components function according to original design specification documents

and functionally comply, as negotiated, with Service Provider

inter-operability requirements.

NPAC Testing Summary January 03, 1997

-Draft-